-- Logs begin at Thu 2022-09-22 11:49:35 UTC, end at Thu 2022-09-22 12:49:21 UTC. -- Sep 22 12:14:38.342462 dv-cinder-856463-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 12:14:38.349742 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 12:14:38 2022] *** Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: nodename: dv-cinder-856463-4-iscsi Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: machine: x86_64 Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: clock source: unix Sep 22 12:14:38.351752 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: pcre jit disabled Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: detected number of CPU cores: 4 Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: current working directory: / Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: detected binary path: /usr/bin/uwsgi-core Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: your processes number limit is 47680 Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: your memory page size is 4096 bytes Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: detected max file descriptor number: 2048 Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: lock engine: pthread robust mutexes Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: thunder lock: enabled Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 12:14:38.352224 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 12:14:38.392827 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: Python main interpreter initialized at 0x56066b51f9c0 Sep 22 12:14:38.392827 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: python threads support enabled Sep 22 12:14:38.392827 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: your server socket listen backlog is limited to 100 connections Sep 22 12:14:38.392827 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: your mercy for graceful operations on workers is 90 seconds Sep 22 12:14:38.393219 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 12:14:38.393301 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: *** Operational MODE: preforking *** Sep 22 12:14:38.393555 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: *** uWSGI is running in multiple interpreter mode *** Sep 22 12:14:38.393555 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: spawned uWSGI master process (pid: 81362) Sep 22 12:14:38.393760 dv-cinder-856463-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 12:14:38.394330 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: spawned uWSGI worker 1 (pid: 81376, cores: 1) Sep 22 12:14:38.395121 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: spawned uWSGI worker 2 (pid: 81377, cores: 1) Sep 22 12:14:38.395121 dv-cinder-856463-4-iscsi devstack@keystone.service[81362]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 12:14:42.653932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 12:14:42.653932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: from cryptography.utils import int_from_bytes Sep 22 12:14:42.654734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 12:14:42.654734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: from cryptography.utils import int_from_bytes Sep 22 12:14:42.655151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 12:14:42.655151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: from cryptography.utils import int_from_bytes Sep 22 12:14:42.655263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 12:14:42.655263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: from cryptography.utils import int_from_bytes Sep 22 12:14:42.704182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 12:14:42.704394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 12:14:42.704589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] command line args: [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 12:14:42.704858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 12:14:42.705035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ================================================================================ {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 12:14:42.705206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 12:14:42.705286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] admin_token = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.705401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 12:14:42.705521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] config_dir = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.705599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] command line args: [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 12:14:42.705708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.705850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 12:14:42.705937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] config_source = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ================================================================================ {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 12:14:42.706087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] debug = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] admin_token = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] config_dir = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.706942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] config_source = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] debug = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_config_append = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.707968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_dir = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_file = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_config_append = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.708846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_dir = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_file = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.709922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.710904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.711037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.711657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.711798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.711929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] publish_errors = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.712923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] publish_errors = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.713986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] standard_threads = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] use_journal = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] standard_threads = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] use_json = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.714959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] use_stderr = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] use_syslog = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] use_journal = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.715751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] use_json = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] use_stderr = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.715942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] use_syslog = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.716146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 12:14:42.716414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.716904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.717881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.718970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719565 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.719925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.720908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.721883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722038 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.722963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.723970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.724938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.725993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.726749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.727990 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.728949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.729998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.730672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731843 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.731937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.732918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.733991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.734933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.735950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.736903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.737855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.738935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.739936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.740888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.external = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.741973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.external = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.password = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.742955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] auth.token = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.password = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] auth.token = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.743973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.744962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.745897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.746848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.747987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.748992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.749944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.750965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.751902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.752870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.753953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.754957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.755909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.756886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.757968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.758949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.759989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.760904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.761880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.762952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.763973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.764961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.765977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.766896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.767858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.768968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.769993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.770925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.771866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.772980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.773060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.773216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.773295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.773463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.773587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.774943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] role.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] role.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.775952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] role.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] role.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.776934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.777910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.778848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.779892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.780963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.781924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.782995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.783930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.784959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.785921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.786869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.787995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.connection = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.788986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.789064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.connection = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.789227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.789355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.790968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.791978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.792937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81376) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 12:14:42.793315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 12:14:42.793941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81377) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 12:14:42.821951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.822178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.822399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.822484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.822694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.822915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.830688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.831220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.859656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.859888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.860204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.860463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.913048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.913299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.961999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.962224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.963562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:42.963795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.117846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.118947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.119029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.119164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.142878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.143719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.155249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.155508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.168971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.169098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.169179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.169312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.169391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81377) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.169525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81376) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 12:14:43.185896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.186106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 12:14:43.186378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.186461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.186589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 12:14:43.186686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 12:14:43.186895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.186975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 12:14:43.187189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.187904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.188161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.197704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.197985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.198225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.198305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.198504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.198593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.198775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.199058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.205427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.205965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.208286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.208592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.208863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.208944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.209968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.210851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.211038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.211117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.211851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.212122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.234165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.234666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.235621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.236116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.236425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.240552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.240840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.241055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.241136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.241342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.241626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.247404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.247690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.247907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.248190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.253238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.253537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.253752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.253836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.254979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.255166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.255246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.255436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.255720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.270996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.271276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.271586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.271865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.275664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.276272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.278878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.279176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.279479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.279777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.283363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.283940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.286482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.287053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.289723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.290956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.291160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.291241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.291449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.291729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.306749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.307039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.307302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.307383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.307564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.307655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.307842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.308104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.314368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.314902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.315900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.316451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.317463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.317742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.317968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.318245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.320435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.321243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.321493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.321575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.321779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.321871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.322061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.322173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.322341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.322733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.333172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.333460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.333722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.333803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.334999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.335217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.335507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.350619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.351229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.353543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.354296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.356678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.357516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.358140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.358936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.359601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.359883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.360273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.360395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.360681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.361078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.366549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.367390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.368019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.368873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.370912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.371311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.371598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.371714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.371874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.372128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.372421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.372701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.379581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.379873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.380156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.380386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.380688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.380969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.387500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81377) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.387787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81376) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 12:14:43.388384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.388954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.389977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390247 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.390866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.391058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81377) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.391137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.391335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.391599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.391866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81376) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 12:14:43.425153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56066b51f9c0 pid: 81377 (default app) Sep 22 12:14:43.425838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56066b51f9c0 pid: 81376 (default app) Sep 22 12:14:43.428349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fb6ea-962f-403b-905d-c2136798adfb None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:43.428638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fb6ea-962f-403b-905d-c2136798adfb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:43.428910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fb6ea-962f-403b-905d-c2136798adfb None None] PATH_INFO: `/v3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:43.433617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Thu Sep 22 12:14:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:14:50.111242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:50.111743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:50.111846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:50.116021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:14:50.121404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:50.121984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:50.121984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:50.191809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: INFO dbcounter [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] Registered counter for database keystone Sep 22 12:14:50.210975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG oslo_db.sqlalchemy.engines [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81377) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 12:14:50.217091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writer thread running {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 12:14:50.400807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG passlib.handlers.bcrypt [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81377) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 12:14:50.401114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG passlib.handlers.bcrypt [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81377) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 12:14:50.458542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:50.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:50.587622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 12:14:51.046990 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:51.116876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: INFO dbcounter [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] Registered counter for database keystone Sep 22 12:14:51.138064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG oslo_db.sqlalchemy.engines [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81376) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 12:14:51.144397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writer thread running {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 12:14:51.261123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2b334-9669-40a5-913f-6afe676bfe7d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:51.265223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:51.265847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:51.266122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:51.266387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:51.274949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:51.325290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 12:14:51.325623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5811165f-1cb2-49cb-bd74-cee4a128fb4f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:51.334794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Sep 22 12:14:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 12:14:51.341252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:51.357979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47248482-685e-4d66-aa00-af514b59fac7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:51.362275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:51.363374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:51.363651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:51.363916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] PATH_INFO: `/v3/projects/8012745f80ac4412b0055fc9da17a750` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:51.365580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] RBAC: Authorizing `identity:get_project(project_id=8012745f80ac4412b0055fc9da17a750)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:51.417550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING keystone.common.rbac_enforcer.enforcer [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 12:14:51.417833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3af71-3ac6-4ab3-9f4f-ab0047a61adc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:51.422296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Thu Sep 22 12:14:51 2022] GET /identity/v3/projects/8012745f80ac4412b0055fc9da17a750 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:53.516734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdad3c-2c3e-493f-8340-761ef679247c None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:53.517254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdad3c-2c3e-493f-8340-761ef679247c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:53.517353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdad3c-2c3e-493f-8340-761ef679247c None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:53.521398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:14:53.526672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:53.527045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:53.527210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:53.547165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:53.548686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:53.556928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 12:14:53.818758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdad3c-2c3e-493f-8340-761ef679247c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:53.831334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdad3c-2c3e-493f-8340-761ef679247c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:53.836154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:53.836780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:53.837061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:53.837333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:53.844899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:53.846660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118beb25-3377-4400-90c8-4fd2de75430e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:53.857371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Thu Sep 22 12:14:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:14:53.863435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:53.872593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aa8cf3-f207-42fb-a11e-a5d32542230f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:53.876902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:53.877525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:53.877804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:53.878072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:53.879364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:53.881233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65361e-f285-4801-b4f4-aef45f7b0f98 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:53.883255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Thu Sep 22 12:14:53 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:14:56.093566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73261539-15ba-4d4a-ab75-b188f4f08c24 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.094117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73261539-15ba-4d4a-ab75-b188f4f08c24 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.094117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73261539-15ba-4d4a-ab75-b188f4f08c24 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.095398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:14:56.101092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.101503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.101628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.106639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b1ffb-72a9-459f-8ca5-3a79e6c5905c None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.106924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b1ffb-72a9-459f-8ca5-3a79e6c5905c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.107200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b1ffb-72a9-459f-8ca5-3a79e6c5905c None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.108448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:14:56.113550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.113831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.114106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.120377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:56.121539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:56.155914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG passlib.handlers.bcrypt [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81376) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 12:14:56.156234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG passlib.handlers.bcrypt [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81376) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 12:14:56.200773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:14:56.205424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:56.207012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:56.307132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:14:56.464777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.488920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:56.567411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f8c7a-019f-401c-8f65-44d0583c0041 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:56.579658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.600798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:56.601431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.601705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.601975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.612758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:14:56.621556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:14:56.622651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:14:56.623719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:14:56.628244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.683929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3e639-8ec2-4a86-a595-8dabe13d2a7c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:56.686226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:56.709318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:56.721778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:56.722580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.722860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.723127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.727435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21f70a-5559-4876-9db6-55c65741d7ba None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:56.760672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:56.761269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.761543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.761809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.762467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:14:56.762767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:56.764144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:56.775972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:56.781550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.789694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:56.791207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:56.879327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e850d-d061-4f22-a4da-e3862cad92b2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:56.892656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:14:56.898634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.901284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:56.916113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:56.916768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:56.917041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:56.917306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:56.918422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:56.919863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:56.921609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:56.926642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:56.940455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:57.001267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1e0798-6326-4f0e-951a-3fe8539164c9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:57.025352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ba229-c878-4ab5-90fe-608c17dde8fc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:57.034532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:57.035367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:57.035639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:57.035904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:57.054252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:57.055707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:57.060097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:57.060756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:57.061033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:57.061302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:57.062399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:57.063834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:57.065569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 10/18] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:57.071722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:14:57.073799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:14:57.074675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:14:57.075471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:14:57.075471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:14:57.075471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:14:57.075471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 9/19] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:14:57.080551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:14:57.136667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:57.153592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:57.171896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2f6bb-c9de-44c3-9405-d7a641d50a8f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:57.179485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034d24b-9fc8-486b-80c2-b23e08787a81 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:14:57.204935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:57.205533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:57.205804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:57.206071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:57.206778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.domain.id=default, user.id=9dcd38c10c5f47bf8d47fab51d035bf9) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:14:57.207096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:57.208551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ed439-28fb-49ce-a8c6-7b110ab7f140 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:57.211817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:14:57.212442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:57.212718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:57.212986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:57.213618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:14:57.213919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:14:57.214966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 11/20] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Sep 22 12:14:57 2022] GET /identity/v3/role_assignments?user.id=9dcd38c10c5f47bf8d47fab51d035bf9&scope.domain.id=default&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:57.215365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37e992-ac42-44b7-a749-b77619481d12 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:14:57.234326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 10/21] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:14:59.437728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57318c-337b-4436-a7b3-b2113b01c287 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:59.438753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57318c-337b-4436-a7b3-b2113b01c287 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:59.439259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57318c-337b-4436-a7b3-b2113b01c287 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:59.441231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 12/22] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:14:59.447373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ab0ac-c80b-4dcb-b5a4-ae355de3fb9e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:59.447845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ab0ac-c80b-4dcb-b5a4-ae355de3fb9e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:59.447948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ab0ac-c80b-4dcb-b5a4-ae355de3fb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:59.478036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a60ab0ac-c80b-4dcb-b5a4-ae355de3fb9e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:59.479221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a60ab0ac-c80b-4dcb-b5a4-ae355de3fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:59.570643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 11/23] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:14:59.621817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6de75-5c8d-43fb-ba3d-0f14eccf24f4 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:59.622464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6de75-5c8d-43fb-ba3d-0f14eccf24f4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:59.622938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6de75-5c8d-43fb-ba3d-0f14eccf24f4 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:59.624486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:14:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:14:59.630009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:14:59.630300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:14:59.630571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:14:59.658543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:14:59.659939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:14:59.722276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:14:59.758187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 12/25] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:14:59.984895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6de75-5c8d-43fb-ba3d-0f14eccf24f4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.113436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6de75-5c8d-43fb-ba3d-0f14eccf24f4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:00.161904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:00.162557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.162859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.163124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.181447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:00.186761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:00.187609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:00.188398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 14/26] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:14:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:00.197568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.267543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.272372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:00.326018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e384d53-4ebe-4da7-bb4a-5784b459e1c6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:00.427607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:00.428221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.429153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.429424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.430088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:00.430391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:00.431769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b816847-9d33-4538-9115-b7eccc4eabee None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:00.455607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 13/27] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:00.461353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c89b9-e2dc-4ef9-a1b8-d0e618bf3f20 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.461633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c89b9-e2dc-4ef9-a1b8-d0e618bf3f20 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.461905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c89b9-e2dc-4ef9-a1b8-d0e618bf3f20 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.464243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 14/28] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:00.464984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c89b9-e2dc-4ef9-a1b8-d0e618bf3f20 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.474122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a3881-3e48-41b9-9dcf-31e24ddb7914 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:00.518337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:00.520781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:00.523074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.523492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.523908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.538412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:00.541020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-66e650c1-ab89-45ee-abbf-e70da6b9d38f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:15:00.561912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:00.562973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:15:00.564033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:15:00.564033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:15:00.564033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:00.565782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 15/29] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:00.566719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b492-12f1-4dcc-9977-88df1bc4b156 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.566995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b492-12f1-4dcc-9977-88df1bc4b156 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.567262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264b492-12f1-4dcc-9977-88df1bc4b156 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.572390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 16/30] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:00.573628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95e5c8-c593-4dff-a253-49d1aa2cb1b1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.574047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95e5c8-c593-4dff-a253-49d1aa2cb1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.574479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95e5c8-c593-4dff-a253-49d1aa2cb1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.592970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c89b9-e2dc-4ef9-a1b8-d0e618bf3f20 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:00.593547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0a95e5c8-c593-4dff-a253-49d1aa2cb1b1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:00.594918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0a95e5c8-c593-4dff-a253-49d1aa2cb1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:00.636161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:00.636822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.637257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.637665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.639789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:00.641890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:00.644273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 15/31] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:00 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:00.645304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.653471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:00.676569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 17/32] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:00.678149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.678633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.679052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.717326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:00.718749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:00.761455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2fb54f-aab4-4a32-b75d-e71aa23f2f70 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:00.769864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:00.801493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:00.802421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:00.802853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:00.803286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:00.805873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:00.806320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:00.808144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:00.820923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 16/33] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:00.822654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:00.845784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 18/34] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:00.846872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:01.005382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be1ef-5f89-40f6-9197-bb90fbd52f9e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:01.060826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:01.061753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.062309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.062751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.065082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:01.066754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:01.068514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 17/35] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:01.075356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:01.091841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef621-0a0a-400f-81a7-1586c88c91f4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:01.119913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:01.133980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:01.173480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:01.174215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.174390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.174668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.175340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:01.176889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:01.197991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2aff57-f88b-4120-9ae7-1e80eeb1adf3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:01.235720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:01.236779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.237227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.237682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] PATH_INFO: `/v3/domains/default/users/9dcd38c10c5f47bf8d47fab51d035bf9/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.246064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9dcd38c10c5f47bf8d47fab51d035bf9, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:01.247918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:01.353106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 19/36] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:15:00 2022] POST /identity/v3/projects => generated 301 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:01.353873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:01.387852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 18/37] 10.222.0.33 () {58 vars in 1361 bytes} [Thu Sep 22 12:15:01 2022] PUT /identity/v3/domains/default/users/9dcd38c10c5f47bf8d47fab51d035bf9/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:15:01.389323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:01.491329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34914f3-6571-45a5-9a14-6c39e20fdd48 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:01.538890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:01.539820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.540244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.540915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.541771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:01.543832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0527bc7-75c2-4069-9063-afbcd45b3a7d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:01.572426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 20/38] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 22 12:15:01 2022] POST /identity/v3/roles => generated 225 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:01.572426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4dac-6d3a-4fa4-a3a0-e9300b91d2b1 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.572426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4dac-6d3a-4fa4-a3a0-e9300b91d2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.572426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c4dac-6d3a-4fa4-a3a0-e9300b91d2b1 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.572426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:01.578404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a587b1-fa73-4633-b635-889042ba1690 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.578713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a587b1-fa73-4633-b635-889042ba1690 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.578968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a587b1-fa73-4633-b635-889042ba1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.609068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-82a587b1-fa73-4633-b635-889042ba1690 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:01.611262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-82a587b1-fa73-4633-b635-889042ba1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:01.635056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:01.642541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa45f19b-347c-4441-9695-1536b74925c5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:01.644015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:01.695603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:01.696237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.696548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.696830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.697676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:01.699555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f2887-e3e6-4402-97e7-c0b0093c280a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:01.707189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 22/40] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:01.730663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 19/41] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 22 12:15:01 2022] POST /identity/v3/roles => generated 219 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:01.865933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a2383-b5de-46e6-ae72-de8133757ad0 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.866445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a2383-b5de-46e6-ae72-de8133757ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.866876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a2383-b5de-46e6-ae72-de8133757ad0 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.868472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 23/42] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:01.875338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:01.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:01.876272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:01.901229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:01.903116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:01.981566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 20/43] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:02.077422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a2383-b5de-46e6-ae72-de8133757ad0 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:02.183294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a2383-b5de-46e6-ae72-de8133757ad0 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:02.218531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:02.219142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:02.219424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:02.219697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:02.220396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:02.221779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:02.237143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 24/44] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 22 12:15:02 2022] POST /identity/v3/roles => generated 223 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:02.254473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:02.347753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:02.367607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718e09e-86cf-499e-86af-2c99b80e32fb None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:02.411202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:02.411870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:02.412167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:02.412480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:02.414275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:02.415183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee481f-7755-4efd-af75-68d56827d09d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:02.416981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 21/45] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:02.422719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:02.549611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:02.631334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1593aaa1-4979-491a-8c88-17e11c411647 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:02.663765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:02.664397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:02.664679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:02.664950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:02.665579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:02.667051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02daf5a-d3d0-45a0-ba14-10d6aeb9df81 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:02.693089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 25/46] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:15:02 2022] POST /identity/v3/projects => generated 312 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:03.728333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f504b1-8d37-43f7-9fdd-fc2b6ee707c4 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:03.728845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f504b1-8d37-43f7-9fdd-fc2b6ee707c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:03.728947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f504b1-8d37-43f7-9fdd-fc2b6ee707c4 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:03.730646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 22/47] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:03.735582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:03.735954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:03.736245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:03.754444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:03.755627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:03.815813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:03.821750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 26/48] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:04.087429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f504b1-8d37-43f7-9fdd-fc2b6ee707c4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:04.288811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f504b1-8d37-43f7-9fdd-fc2b6ee707c4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:04.321507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:04.322135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.322407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.322686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.333580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:04.339902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:04.340765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:04.341511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 23/49] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:04.348097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:04.444670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7493d5-966b-4ac2-90fa-38b14039e3c1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:04.477451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:04.478054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.478332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.478613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.479259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:04.479813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:04.481227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:04.489239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 27/50] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:04.494981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:04.510302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:04.592475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a4083-5d28-454a-af20-de4489141827 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:04.625250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:04.625878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.626161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.626437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.627564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:04.629054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:04.630844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 24/51] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:04 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:04.636141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:04.695146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:04.734564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf8f7d-c6d8-45c0-9524-db89dfd7e24d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:04.770753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:04.771371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.772211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.772523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.773631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:04.775088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1966e3d-e422-4f46-9cfd-b48f4dd2a11a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:04.776900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 28/52] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:04.782486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:04.855650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17205-9ce3-46d4-b3ab-3cd9414ea036 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.855856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17205-9ce3-46d4-b3ab-3cd9414ea036 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.856134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e17205-9ce3-46d4-b3ab-3cd9414ea036 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.857528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 29/53] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:04.862772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.863054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.863328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.868875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:04.882976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:04.884188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:04.890181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07f1e6-73a5-4294-97dd-d4dbd0813b63 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:04.923243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:04.923845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:04.924124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:04.924429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:04.925151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.domain.id=default, user.id=9dcd38c10c5f47bf8d47fab51d035bf9) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:04.925458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:04.926900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:04.934651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 25/54] 10.222.0.33 () {56 vars in 1322 bytes} [Thu Sep 22 12:15:04 2022] GET /identity/v3/role_assignments?user.id=9dcd38c10c5f47bf8d47fab51d035bf9&scope.domain.id=default&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:04.951294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 30/55] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:05.221533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:05.316893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575a94c-3f47-4620-8d29-dd6175029f35 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:05.348968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49880545-c930-471c-a4d6-148671c56724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:05.349571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880545-c930-471c-a4d6-148671c56724 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:05.350001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880545-c930-471c-a4d6-148671c56724 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:05.350276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880545-c930-471c-a4d6-148671c56724 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:05.351357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49880545-c930-471c-a4d6-148671c56724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:05.352822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49880545-c930-471c-a4d6-148671c56724 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:05.354319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 26/56] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:05.359984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:05.415889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:05.455260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf086b-c61d-4c84-ab3f-f68647888274 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:05.486791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:05.487336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:05.487610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:05.487877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:05.488528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:05.489962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff155e-76d5-490a-8b11-45a9fdf89c6b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:05.515654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 31/57] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Sep 22 12:15:05 2022] POST /identity/v3/projects => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:07.651892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ec9a1-3b48-49a5-a218-8a687f3b9cba None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:07.652429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ec9a1-3b48-49a5-a218-8a687f3b9cba None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:07.652509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ec9a1-3b48-49a5-a218-8a687f3b9cba None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:07.653804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 27/58] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:07.658935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:07.659347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:07.659479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:07.677048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:07.678767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:07.741909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 32/59] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:08.007930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ec9a1-3b48-49a5-a218-8a687f3b9cba None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:08.022884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:08.106965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ec9a1-3b48-49a5-a218-8a687f3b9cba None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:08.139191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:08.139807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:08.140087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:08.140389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:08.141495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:08.142932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a1bc1-a71b-4a74-b870-b4a88a67cba7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:08.144503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 28/60] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:08.150520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:08.191001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:08.246279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d37c0-59b0-4039-a99a-a7ce4518feb8 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:08.278958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:08.279499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:08.279780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:08.280054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:08.280732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:08.282186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7fc78-9137-45e4-ba08-62b3daeec1ea None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:08.308271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 33/61] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:08 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:11.438044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98569af-f68b-4df2-b66c-cbf8b4f68f14 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:11.439047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98569af-f68b-4df2-b66c-cbf8b4f68f14 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:11.439530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98569af-f68b-4df2-b66c-cbf8b4f68f14 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:11.441125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 29/62] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:11.446607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:11.447383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:11.447856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:11.466656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:11.468160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:11.550622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 34/63] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:11.890204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b7506-6c45-417e-8918-22664fed77f1 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:11.890786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b7506-6c45-417e-8918-22664fed77f1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:11.891283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7b7506-6c45-417e-8918-22664fed77f1 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:11.892882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 30/64] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:11.894830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:11.898701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390aef-a6ef-4718-a820-6b013e2b923e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:11.899211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390aef-a6ef-4718-a820-6b013e2b923e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:11.899709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83390aef-a6ef-4718-a820-6b013e2b923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:11.926596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-83390aef-a6ef-4718-a820-6b013e2b923e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:11.929372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-83390aef-a6ef-4718-a820-6b013e2b923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:11.956705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.009168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 31/65] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:12.010935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec42f1-3972-4a87-9815-0dface6a63d8 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.011446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec42f1-3972-4a87-9815-0dface6a63d8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.011913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec42f1-3972-4a87-9815-0dface6a63d8 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.013475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 32/66] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:12.024379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f211d25-2f1e-416d-9300-22d5dd1ce4a1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.025131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f211d25-2f1e-416d-9300-22d5dd1ce4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.025720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f211d25-2f1e-416d-9300-22d5dd1ce4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.033685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9284dd3-1d2a-49a2-b04d-4fdcbc5b215c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.059476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3f211d25-2f1e-416d-9300-22d5dd1ce4a1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:12.068537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.069174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.069442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.069712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.070198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3f211d25-2f1e-416d-9300-22d5dd1ce4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:12.073588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.081360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:12.087617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:12.088472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:12.089256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:12.091741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 35/67] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:12.109168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.157592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 33/68] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:12.232520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88e1d-bc11-4839-8097-1d3e12829e54 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.232830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88e1d-bc11-4839-8097-1d3e12829e54 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.233095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f88e1d-bc11-4839-8097-1d3e12829e54 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.234614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 34/69] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:12.251605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.251902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.252162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.281226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:12.282460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:12.296702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.311109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477831-6cb0-4672-9219-7a85c94431f4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.350131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.350822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.351098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.351368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.352026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:12.352347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:12.354472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:12.364536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 36/70] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:12.364843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 35/71] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 22 12:15:12.366572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.371401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.451480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.477271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a878f-42e3-4b8b-99f3-9573b374e1d8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.484612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7331801-fed8-431b-8702-689d4f3b8ea4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.518364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.518994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.519272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.519543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.521611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.522243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.522523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.522793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.531103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0ac5cbe0-6d72-4ffe-9833-1bf87b271343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:12.538189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:12.539057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:12.545620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:12.546192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:12.546713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 37/72] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:12.546817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:12.546886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8030b-d4c2-4ec9-9229-ec35d270604d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.546886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8030b-d4c2-4ec9-9229-ec35d270604d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.546886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8030b-d4c2-4ec9-9229-ec35d270604d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.547068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 36/73] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:12.547068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.547191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 38/74] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:12.547433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8030b-d4c2-4ec9-9229-ec35d270604d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.562771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.650801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9581cd-8bbb-4f48-908a-7aa646ba121e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.672129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8030b-d4c2-4ec9-9229-ec35d270604d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.677647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.685752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.686373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.686656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.686930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.696893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-202f2ce2-32ff-4531-8019-a278e0bb1770 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:12.701692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:12.702615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:12.703574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 37/75] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:12.704411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.704411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.704411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.709927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.710543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.710825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.711110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.711772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:12.712080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:12.713501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:12.721889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:12.722112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 39/76] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:12.722879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.723070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:12.788398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 38/77] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:12.789234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:12.796915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.826047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552db1a-6a93-4300-933d-b157a683c884 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.862480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.863508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.863786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.864062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.898477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c963ee2f-518f-4b45-814a-6e1c6da507cc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:12.918414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:12.932445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:12.934096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:12.936775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:12.937404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:12.937684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:12.937957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:12.939065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:12.940516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:12.942482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 39/78] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:12.943107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.005925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.037657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 40/79] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:13.038323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.054788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2aa42-1646-490f-bca9-28b64203498a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.088967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.089997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.090483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.090954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.092154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:13.092724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.094867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.103398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 40/80] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.104449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.138459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924e836-36ad-450b-8e73-60bc8e0e55e5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.152977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.172119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.172772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.173046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.173315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.174408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.175878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.177495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 41/81] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.178331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.205566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9432ac9-adb7-49ee-b4e4-2587cba75df6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.218667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.238857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.239467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.239774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.240050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.240721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.242172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.269981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 41/82] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:13 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:13.270717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.291028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88574c-bee1-4f3a-80c3-13d16b1d5dcd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.329737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.330391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.330716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.330935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.332202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.334437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.341030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 42/83] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.341030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.392121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de5b87-0f04-4d2c-83ba-f91ae8f204a8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.395866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.437822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.438486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.438765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.439039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.445563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf68496-e023-4116-8df5-9e635e5fd672 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.452477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:13.455221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:13.459295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:13.460135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 42/84] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:13.460948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.484720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.485671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.485946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.486222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.487350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.489286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.491407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 43/85] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.492171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.587521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d2e0b-e402-4a2c-956a-8478f9ceaebf None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.611094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e117b-ad3f-40be-8ca8-291940f0277c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.630199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.631517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.631870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.632160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.649355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.649968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.650243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.650621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.651227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=6c34613c69674b59957862a501334be0) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:13.651529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.653009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.660160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 44/86] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.665155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.671358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.719795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.722351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.735276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.801776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f514f7-1c8d-4fcc-b5dc-9ba728b27b3f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.840418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:13.841040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:13.841324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:13.841600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:13.842276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:13.842621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:13.844190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:13.852776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 43/87] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:13.854268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.854654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 45/88] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:13.860140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:13.928115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:13.976128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4446-df27-429c-8ce5-0ebd6fa0a871 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:13.997917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e5aa5-fc7e-46e4-8328-6cb7ab4fda6e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.000523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:14.032087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.032747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.033024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.033296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.058557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.059195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.059472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.059743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.061914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.063380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.072639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 46/89] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:14.073528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:14.163880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.166131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.264653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9432c-6c80-4de3-a8ab-ac7da8f08f4a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.278191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:14.329246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.329895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.330173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.330443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.331732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.333207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.338405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 47/90] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:14 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:14.339076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:14.346988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 44/91] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:14.347827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:14.381998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:14.447216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420b653-9122-4f43-b43a-4aff139b2d95 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.497646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.499066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.499643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.500110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.501041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=9dcd38c10c5f47bf8d47fab51d035bf9) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:14.501559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.503187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.509807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 48/92] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:14 2022] GET /identity/v3/role_assignments?user.id=9dcd38c10c5f47bf8d47fab51d035bf9&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:14.510892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:14.600014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f8974-bf05-416b-8754-f6b70552f74f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.620617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac5a68-87f1-4b90-9479-45f7ea76289d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.632662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:14.665098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.666093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.666568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.667033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.668493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.670344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.672378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 49/93] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:14 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:14.682412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:14.696989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.697519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.697742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.698014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.797124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc5dcf-40e6-4a20-aa47-fc2c8f1bffe1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:14.805905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:14.838786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:14.839753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:14.840228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:14.840814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:14.841726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=6c34613c69674b59957862a501334be0) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:14.842242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.843890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:14.851125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 50/94] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:14 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=49615db920c64505958165b68049a95e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:14.919115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:14.920690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:15.061931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 45/95] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 714 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:15.067578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:15.119741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:15.196993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea399eba-f05f-4c50-8db6-2a947e014e22 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:15.231377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:15.232013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:15.232312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:15.232591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] PATH_INFO: `/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:15.233717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] RBAC: Authorizing `identity:get_project(project_id=e5f5a288fd234f50b48bef1c0f210ab4)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:15.235175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6554e-d149-4001-ae9d-b8688c4e49ee None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:15.236829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 51/96] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:15 2022] GET /identity/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:15.242984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:15.331973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:15.382694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3777a-311c-4a6b-83f7-cdd90b70e63f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:15.426242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:15.426852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:15.427155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:15.427429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:15.428144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=e5f5a288fd234f50b48bef1c0f210ab4, user.id=6c34613c69674b59957862a501334be0) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:15.428481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:15.429930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fea4d-882f-4bc4-9176-ea44bbb3f796 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:15.440454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 46/97] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:15 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=e5f5a288fd234f50b48bef1c0f210ab4&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:16.072585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334de0-2efc-4308-b88f-58da4372490b None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.073609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334de0-2efc-4308-b88f-58da4372490b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.074110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99334de0-2efc-4308-b88f-58da4372490b None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.075645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 52/98] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:16.081202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3fa8-de23-4a2e-a446-733b196b2035 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.082048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3fa8-de23-4a2e-a446-733b196b2035 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.082525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22f3fa8-de23-4a2e-a446-733b196b2035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.103081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d22f3fa8-de23-4a2e-a446-733b196b2035 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:16.104897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d22f3fa8-de23-4a2e-a446-733b196b2035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:16.176988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 47/99] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:16.444006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99334de0-2efc-4308-b88f-58da4372490b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:16.558138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99334de0-2efc-4308-b88f-58da4372490b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:16.560079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:16.563837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d75b58-937c-4d62-9a18-f24ebd27cb65 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.564120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d75b58-937c-4d62-9a18-f24ebd27cb65 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.564419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d75b58-937c-4d62-9a18-f24ebd27cb65 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.565774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 48/100] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:16.573758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd37d-900f-434b-a2e7-d50db0f89cb8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.574167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd37d-900f-434b-a2e7-d50db0f89cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.574938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd37d-900f-434b-a2e7-d50db0f89cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.596472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-fabfd37d-900f-434b-a2e7-d50db0f89cb8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:16.597963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fabfd37d-900f-434b-a2e7-d50db0f89cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:16.617124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:16.617773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.618049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.618317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.619544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:16.621124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:16.622690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 53/101] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:16.641511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:16.667455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:16.685806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 49/102] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:16.748666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097cae-43ad-43c8-a239-9344566e5532 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.749261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097cae-43ad-43c8-a239-9344566e5532 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.749734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c097cae-43ad-43c8-a239-9344566e5532 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.751269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 50/103] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:16.759769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.760369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.760858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.792221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:16.793779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:16.811350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9556ab9-59ae-4a9c-a977-6f9d9460a856 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:16.856185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:16.856906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:16.857200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:16.857569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:16.858221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:16.859680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:16.890915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 54/104] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:15:16 2022] POST /identity/v3/projects => generated 302 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:16.946429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 51/105] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:17.126581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.152753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:17.242117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ea39a-f429-46b7-8853-6679f83cbffc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:17.287540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:17.288535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.289011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.289470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.305708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:17.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:17.311663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:17.312492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:17.314723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 55/106] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:17.320656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.439352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f272ba-6900-43f6-8fbc-ca583fe5e46b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:17.457740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:17.475470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:17.476083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.476396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.476674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.477343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:17.477645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:17.479295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:17.488057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 52/107] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:17.493923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.578306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.617538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:17.653211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f283375-6981-4694-ac46-2d80c71a7a56 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:17.726139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:17.726712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.726987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.727259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.728656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:17.734185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:17.734185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 56/108] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:17.749115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.784446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892964f-4cfa-4e9e-af7f-d4639106301f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:17.851779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:17.852454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.852764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.853071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.872485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b3cbef66-9305-4c8f-9686-c6f4ec1bb124 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:17.881888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:17.882752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 53/109] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:17.883515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841ae2-94b3-4bdb-b60c-b73a4287ab9e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.884285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841ae2-94b3-4bdb-b60c-b73a4287ab9e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.884285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29841ae2-94b3-4bdb-b60c-b73a4287ab9e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.884285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 54/110] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:17.889282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29841ae2-94b3-4bdb-b60c-b73a4287ab9e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:17.929198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498e7a2-b8e3-4964-b6db-2f918495920e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:17.967831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:17.990541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:17.991147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:17.991476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:17.991753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:17.993600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:17.995857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222edcd-298b-4b36-8673-03c26aa85d03 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:17.998944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 57/111] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:18.000123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.005971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.006266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.042445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:18.046360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:18.075458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:18.082544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29841ae2-94b3-4bdb-b60c-b73a4287ab9e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:18.126004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:18.126623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.126906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.127183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.128027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:18.128363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:18.130840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:18.141162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 55/112] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:18.141626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:18.169871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 58/113] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:18.170746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:18.274986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d52af0-ac08-4507-bb53-0a9ea6ff9f71 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:18.286584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:18.316887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:18.318373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.318842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.319250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/6c34613c69674b59957862a501334be0/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.321519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, user_id=6c34613c69674b59957862a501334be0, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:18.328316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:18.343256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 56/114] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:18 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/6c34613c69674b59957862a501334be0/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:18.429521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f45f-53d9-4dcd-884f-f7e367f5598c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:18.477071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:18.477734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.478016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.478293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] PATH_INFO: `/v3/users/0bce92399ffd4a4cb1398f51eda5dae7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.493376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] RBAC: Authorizing `identity:get_user(user_id=0bce92399ffd4a4cb1398f51eda5dae7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:18.494858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:18.496611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 59/115] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:18 2022] GET /identity/v3/users/0bce92399ffd4a4cb1398f51eda5dae7 => generated 289 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:18.502806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:18.638389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182b07-3a4c-48fb-80db-996f011112d4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:18.685281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:18.685914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.686195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.686478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.705930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:18.718536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:18.737800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:18.800646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:18.802160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1758ab-59c2-41f1-8622-eb5704698334 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:18.855991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58fc0-fa26-4833-89b8-44fb0e8580de None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:18.912420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:18.912985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.913678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.914117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.923996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 57/116] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:18.925521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f157e55-c9f4-487a-95f7-dc22b9f1bdfe None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.926003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f157e55-c9f4-487a-95f7-dc22b9f1bdfe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.926503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f157e55-c9f4-487a-95f7-dc22b9f1bdfe None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.927859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:18.928483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 58/117] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:18.929679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d38176-2265-4b41-b24f-83fd08343e86 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.930178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d38176-2265-4b41-b24f-83fd08343e86 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.930637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d38176-2265-4b41-b24f-83fd08343e86 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.931977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 59/118] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:18.936808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d38176-2265-4b41-b24f-83fd08343e86 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d37bc09e-c9e5-4c26-bb7f-f31f8777a41a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:18.936929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:18.937717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:18.938494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 60/119] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:18.939617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab1bd-dc23-437a-9c80-3ba255192d52 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:18.939717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab1bd-dc23-437a-9c80-3ba255192d52 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:18.939990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eab1bd-dc23-437a-9c80-3ba255192d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:18.963348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a9eab1bd-dc23-437a-9c80-3ba255192d52 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:18.964603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a9eab1bd-dc23-437a-9c80-3ba255192d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:19.027076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.031752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.034699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 61/120] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:19.040945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.040945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.040945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.069219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:19.070495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:19.114017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d38176-2265-4b41-b24f-83fd08343e86 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.177189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.177737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.178016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.178286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.179651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:19.181126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:19.182755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 60/121] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:18 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:19.183502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.196645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 62/122] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:19.197305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.388186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36febb5-2b8f-41ab-a0b7-3f351106b14c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.392044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c719a07a-ad5e-4bff-bb08-5884a569fa1d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.418442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.428614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.429610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.429900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.430183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.430915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=0bce92399ffd4a4cb1398f51eda5dae7) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:19.431230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:19.432932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:19.439227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 63/123] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/role_assignments?user.id=0bce92399ffd4a4cb1398f51eda5dae7&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:19.440281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.459259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.477067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.479110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.479388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.479663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.480364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:19.480673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:19.482097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:19.505591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 61/124] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:19.511161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.638788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b63af-0c81-4f3c-84ea-b94e9405fa5a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.714566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.715129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.715407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.715679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.728139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082504a9-9798-4ec0-b85c-9c187168f5d3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.729790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:19.737059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:19.737907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:19.738697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:19.742314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 64/125] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:19.743979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.767476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.768508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.768979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.769511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.770766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:19.772418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:19.774225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 62/126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:19.775072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:19.813568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.844905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:19.894294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930ec59-9457-4ca6-9a85-4a3598d7f807 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.911631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedd20a-d9cb-41ea-a07d-171e339962db None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:19.930310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:19.931246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:19.931718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:19.932221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:19.933224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:19.933717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:19.935271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:19.944036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 63/127] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:19.945197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.013441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.013840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.014129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.014413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.045614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:20.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:20.052550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:20.053420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:20.058748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28550989-58b5-4cd2-97ef-3f10175a9808 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.062214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 65/128] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:20.063080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.100538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.101553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.102023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.102498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.104060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.105725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.107578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 64/129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:19 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.108099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.109503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.188459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0b4ea-d369-4e86-a77d-4890794ff0df None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.200519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.226264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.227312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.227781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.228242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.229839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.230057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b239912-89e8-47f3-a3bf-7f0c72010251 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.231771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.233670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 66/130] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:20 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.234587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.270028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.271141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.271428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.271701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.272651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:20.272959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.275232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b6e41d-3da4-423b-b216-4eb7395a01c1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.290888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 65/131] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.290888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661588eb-8634-4725-8b10-1e6e76491675 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.290888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661588eb-8634-4725-8b10-1e6e76491675 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.290888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661588eb-8634-4725-8b10-1e6e76491675 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.290888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 66/132] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:20.294211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661588eb-8634-4725-8b10-1e6e76491675 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.349926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb959b-f5c8-43b5-b48c-2ac29ce45b2a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.358497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.389670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.390825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.391325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.391879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/9dcd38c10c5f47bf8d47fab51d035bf9/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.394269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, user_id=9dcd38c10c5f47bf8d47fab51d035bf9, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.396608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.417073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 67/133] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:20 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/9dcd38c10c5f47bf8d47fab51d035bf9/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:20.418602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.419068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661588eb-8634-4725-8b10-1e6e76491675 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.480555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.501163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.501850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.502131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.502415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.503720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.506049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3cbcb-9666-4ac1-8aa5-425a67066dd7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.507943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 67/134] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:20 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.513800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.514144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.514368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.562956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727737-086d-41b1-ae99-1cc26f30b4b9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.563489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:20.564989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:20.642541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.643177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.643461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.643736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.648399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.649838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.651447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 68/135] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:20 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.653649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.669101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 68/136] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:20.670036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.761082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.840387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb4570-9d31-4a03-977e-5be70466229d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.851524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:20.853929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9c1ca-6ef2-4dd2-aaf5-88edc0b0ca32 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:20.905262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.905934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.906213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.906489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/6c34613c69674b59957862a501334be0/roles/49615db920c64505958165b68049a95e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.909701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, user_id=6c34613c69674b59957862a501334be0, role_id=49615db920c64505958165b68049a95e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.911211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.918059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:20.918733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:20.919022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:20.919338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] PATH_INFO: `/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:20.920627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] RBAC: Authorizing `identity:get_project(project_id=e5f5a288fd234f50b48bef1c0f210ab4)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:20.922161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:20.924011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 69/137] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:20 2022] GET /identity/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:20.931899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:20.933781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 69/138] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:20 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/6c34613c69674b59957862a501334be0/roles/49615db920c64505958165b68049a95e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:21.008673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:21.102243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad5e48d-e752-4e47-b5c4-94613d79aac9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:21.140085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:21.177105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c26ae-271a-4258-85a6-1f1a71ec7646 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:21.179364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:21.180041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:21.181582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:21.181582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] PATH_INFO: `/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4/users/6c34613c69674b59957862a501334be0/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:21.184494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5f5a288fd234f50b48bef1c0f210ab4, user_id=6c34613c69674b59957862a501334be0, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:21.188326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9140e4-0caf-40d7-acbd-a903bbe1d160 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:21.218298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 70/139] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:20 2022] PUT /identity/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4/users/6c34613c69674b59957862a501334be0/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:21.223468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f778a-7365-454a-8cc5-cd3c090cdb32 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:21.223468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f778a-7365-454a-8cc5-cd3c090cdb32 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:21.223468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f778a-7365-454a-8cc5-cd3c090cdb32 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:21.223468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 71/140] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:21.242568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:21.242886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:21.243190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:21.243937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:21.245160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:21.245638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:21.246089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:21.249044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:21.250774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:21.257000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 70/141] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:21.273375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:21.292332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:21.292800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:21.355068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:21.426407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc212-9681-4da4-9cc9-2f9571a4a11d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:21.463910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:21.464968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:21.465343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:21.465708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:21.466935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:21.472074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 72/142] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:21.473900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:21.519122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 71/143] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:21 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:21.830495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:21.971308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd22f-ce06-4964-b02a-60b2383bc838 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:21.986380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:22.016888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:22.017521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.017799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.018070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.037154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:22.041992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:22.042895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:22.043769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 73/144] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:22.049266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:22.176256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:22.189885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5646757-7427-4b0b-81b4-76f7efafc3e4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:22.238189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:22.238818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.239128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.239399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.240070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:22.240385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:22.241773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61111988-cdba-46bb-aecd-4eb7e7763bed None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:22.256785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 72/145] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:22.264010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:22.293364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab49404-be19-41c2-b2ba-e223754452a4 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.293669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab49404-be19-41c2-b2ba-e223754452a4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.293932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab49404-be19-41c2-b2ba-e223754452a4 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.295542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 73/146] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:22.300382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.300842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.301122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.322841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:22.324036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:22.443428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f77fa9-7ea9-4a8f-92dc-1de81de52322 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:22.454240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 74/147] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:22.488558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:22.489405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.489749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.490091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.491369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:22.492980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:22.494840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 74/148] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:22 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:22.500442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:22.615307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:22.635418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edfbe3f-67bb-4e61-85cc-6f4ec24ed4ef None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:22.684097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:22.684729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:22.685004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:22.685270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:22.769702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:22.771802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:22.887489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:22.907743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 75/149] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:22.913404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:22.971461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:23.025653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.058500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9bb893-532a-4db3-9a6f-3ef562d36f60 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.095704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.096714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.097227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.097690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.099061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:23.100733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:23.102653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 75/150] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:22 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:23.109540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.200771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f215b2-40ec-419f-8742-dfe11d0f8d53 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.251869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.252898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.253469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.253954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.267068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:23.273008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:23.273853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:23.274622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:23.275210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1383da2-5d49-41af-8163-ddb216b32bfb None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.276210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:23.278214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 76/151] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:23.284810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.326212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.326822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.327099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.327367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.328098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=6c34613c69674b59957862a501334be0) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:23.328448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:23.329880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:23.342040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 76/152] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:23 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:23.368748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:23.436059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5929e-aaaa-4267-9d83-f2416c18cedc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.478814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.479430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.479702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.479971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.480664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:23.480964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:23.482359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:23.496313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 77/153] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:23.502136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.628567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a0d3c-90ce-4204-a555-28211f96b21d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.671784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.672450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.672729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.672999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] PATH_INFO: `/v3/users/0bce92399ffd4a4cb1398f51eda5dae7` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.674173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] RBAC: Authorizing `identity:get_user(user_id=0bce92399ffd4a4cb1398f51eda5dae7)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:23.675644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:23.678202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 77/154] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:23 2022] GET /identity/v3/users/0bce92399ffd4a4cb1398f51eda5dae7 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:23.683674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.799789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d106c-5198-4131-a6b3-7c742376c2a9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:23.805327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:23.837856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:23.838490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:23.838763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:23.839033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:23.845021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:23.846483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab4f5f5-e3e6-4b6b-a59b-b7ac6e5f77e3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:23.852375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 78/155] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:23 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:23.855178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:23.901502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:23.977088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50701d7-be90-4612-a7a5-29bfc12c41e3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:24.021523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:24.022191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:24.022467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:24.022736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/0bce92399ffd4a4cb1398f51eda5dae7/roles/543bbe978c784ef38f0c36d95b5a194b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:24.024732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, user_id=0bce92399ffd4a4cb1398f51eda5dae7, role_id=543bbe978c784ef38f0c36d95b5a194b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:24.026190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58ec6d-de4c-4b69-b0e7-9459c3a3dab9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:24.048182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 78/156] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:23 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/users/0bce92399ffd4a4cb1398f51eda5dae7/roles/543bbe978c784ef38f0c36d95b5a194b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:24.830713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d895c6-ac73-4724-96a1-ef0e970b60d9 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:24.831254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d895c6-ac73-4724-96a1-ef0e970b60d9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:24.831254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d895c6-ac73-4724-96a1-ef0e970b60d9 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:24.832650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 79/157] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:24.838332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039ebf-0ad6-4323-9c3d-fb6e00246e6c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:24.838798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039ebf-0ad6-4323-9c3d-fb6e00246e6c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:24.838872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1039ebf-0ad6-4323-9c3d-fb6e00246e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:24.864690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ff2e0-4f33-4daf-b57b-e771a8fbcf24 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:24.865037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ff2e0-4f33-4daf-b57b-e771a8fbcf24 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:24.865226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ff2e0-4f33-4daf-b57b-e771a8fbcf24 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:24.866618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 80/158] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:24.866805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d1039ebf-0ad6-4323-9c3d-fb6e00246e6c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:24.870283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d1039ebf-0ad6-4323-9c3d-fb6e00246e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:24.875867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33811976-fe4e-4c1f-a6bf-3d920da7e20f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:24.875867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33811976-fe4e-4c1f-a6bf-3d920da7e20f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:24.875867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33811976-fe4e-4c1f-a6bf-3d920da7e20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:24.912071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-33811976-fe4e-4c1f-a6bf-3d920da7e20f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:24.913331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-33811976-fe4e-4c1f-a6bf-3d920da7e20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:25.038000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 79/159] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:25.062585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:25.113464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc688a7-ccf5-4a8c-bf2a-14d2f37909ed None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:25.114645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc688a7-ccf5-4a8c-bf2a-14d2f37909ed None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:25.115134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc688a7-ccf5-4a8c-bf2a-14d2f37909ed None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:25.116798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 80/161] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:25.130161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:25.130437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:25.130897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:25.157344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:25.158774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:25.244055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 82/162] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:25.575082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc688a7-ccf5-4a8c-bf2a-14d2f37909ed None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:25.615372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:25.616433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:25.674909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:25.761523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc688a7-ccf5-4a8c-bf2a-14d2f37909ed None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:25.811134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0334169-c49a-43c8-a97d-4c1149a34d51 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:25.838868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:25.840255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:25.840761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:25.841221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:25.858970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:25.864951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:25.866021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:25.867057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:25.872788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 81/163] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:25.874492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:25.878541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:25.880067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:25.880947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:25.881280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:25.903388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:25.909722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:25.911173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:25.911948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 83/164] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:25.913159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.017867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.029280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc8686-1e7a-4707-aaa7-3482e5e7a158 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.049369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.070359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.070967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.071302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.071575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.072272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:26.072601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:26.074517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f740c-ef88-46ae-ad04-39a141ffa7db None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:26.081831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51270c0a-600c-459c-8370-7e5a5e7e0ae5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.083634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 84/165] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:26.085890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831de84-99d3-4edc-a466-ec8fe9911841 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.086350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831de84-99d3-4edc-a466-ec8fe9911841 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.086780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831de84-99d3-4edc-a466-ec8fe9911841 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.088170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 85/166] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:26.089379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831de84-99d3-4edc-a466-ec8fe9911841 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.148058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.148733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.149007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.149332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.167221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:26.176121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:26.177019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:26.177819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 82/167] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:26.178548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.266070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831de84-99d3-4edc-a466-ec8fe9911841 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.334160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.334774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.335045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.335311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.335968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:26.336265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:26.337660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36efc9e7-0042-448a-b563-ed29fe9e1fc3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:26.352090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 86/168] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:26.353685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.354071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.354347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.361673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b775c1-26cc-4807-8ecc-2d100c4c5cc8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.379816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:26.381052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:26.392956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.396778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.397458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.397735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.398004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.399882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:26.406631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:26.406631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 83/169] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:26.406631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.437109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.505562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 87/170] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:26.506506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.665284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378bb9e2-b62d-4d6d-ac11-4f74e926740f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.695379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e64b1b-92a4-4c71-a24a-20f2b7ccaa4a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.763119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.763777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.764052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.764337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] PATH_INFO: `/v3/users/6c34613c69674b59957862a501334be0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.769189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] RBAC: Authorizing `identity:get_user(user_id=6c34613c69674b59957862a501334be0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:26.772103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:26.774273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 88/171] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/users/6c34613c69674b59957862a501334be0 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:26.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.793076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:26.793710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:26.793989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:26.794261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:26.794937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:26.795236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:26.796657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:26.800188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.815095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 84/172] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:26.815951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:26.940483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:26.976612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c825f-646d-4af4-bf12-6b57e657eb04 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:26.978118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e819c-d859-4781-bf30-de03e51a6542 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.014726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.015331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.015445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.015599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.015864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.016030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.016322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.016599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.090569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.092018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a535c6-77ff-484a-b4df-877a78f0bc0c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.092333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.093789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.106824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.171616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.202505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 85/173] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:27.202505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.202764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 89/174] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:27.202764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe372-80b7-447e-96e3-92005fb95a2d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.202764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe372-80b7-447e-96e3-92005fb95a2d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.202764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfe372-80b7-447e-96e3-92005fb95a2d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.202764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 90/175] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:27.204108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80017-c738-469a-88a8-656ddf9d27d9 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.204108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80017-c738-469a-88a8-656ddf9d27d9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.208468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc80017-c738-469a-88a8-656ddf9d27d9 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.208468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 91/176] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:27.208468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80017-c738-469a-88a8-656ddf9d27d9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.313521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f99aba-f5ac-4f1f-8e9a-00686f9a9f95 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.317425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc80017-c738-469a-88a8-656ddf9d27d9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.331126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.346345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.346966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.347240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.347512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] PATH_INFO: `/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.350029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] RBAC: Authorizing `identity:get_user(user_id=9dcd38c10c5f47bf8d47fab51d035bf9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.351469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.351766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.352420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.352691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.352956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.353077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 86/177] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/users/9dcd38c10c5f47bf8d47fab51d035bf9 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:27.354090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.366707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:27.372738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:27.373780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:27.374740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 92/178] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:27.375776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.408712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.461754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca57e-7a53-4067-8ab4-b2071fc02b84 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.471944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d24a54-8d8c-4edc-be78-0e320daf62ff None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.496088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.496734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.497009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.497288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.498390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.499819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2f95b2-21cc-4eaf-94cd-a0882353e16d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.501478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 87/179] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:27.502553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843532-c57a-420f-a8c0-da66a005c948 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.502825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843532-c57a-420f-a8c0-da66a005c948 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.503096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0843532-c57a-420f-a8c0-da66a005c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.506622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.507379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.507687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.508045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] PATH_INFO: `/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.509432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5f5a288fd234f50b48bef1c0f210ab4)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.510863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29eb2eb-0c8d-44c7-891f-f861cc00f0b0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.512486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 93/180] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/projects/e5f5a288fd234f50b48bef1c0f210ab4 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:27.513675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.513953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.514217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.522578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e0843532-c57a-420f-a8c0-da66a005c948 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:27.523783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e0843532-c57a-420f-a8c0-da66a005c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:27.532217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:27.533378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:27.567307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.590052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 88/181] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:27.591287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39a77-66db-4352-a3d0-bf3324470af5 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39a77-66db-4352-a3d0-bf3324470af5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.591838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39a77-66db-4352-a3d0-bf3324470af5 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.593326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 89/182] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:27.593851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e39a77-66db-4352-a3d0-bf3324470af5 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.616319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 94/183] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:27.616319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.646048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.715287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e39a77-66db-4352-a3d0-bf3324470af5 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.746851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33a6b3-86eb-48cf-ab05-f309d1026574 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.759829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.760456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.760730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.760997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.784837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.786277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.786697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.787238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.788129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:27.788589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.790123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.799069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 95/184] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:27.804660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.833221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.834738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.885986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:27.913532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7027b6-0f8c-4ee0-bbfc-02bfbeac4221 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:27.942315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 90/185] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:27.943205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:27.949238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:27.949840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.950116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.950385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.951592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=6c34613c69674b59957862a501334be0) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:27.951896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:27.953734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8f0f1-0bea-43e2-b243-8f4537ade6fb None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:27.960355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 96/186] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=49615db920c64505958165b68049a95e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:27.961464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:27.961742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:27.962008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:27.986450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:27.987698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:28.051405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e43f2c8-6169-4aed-8c40-dcc73831466a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.068810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 97/187] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:28.069695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.099175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.100430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.101012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.101482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.102420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=e5f5a288fd234f50b48bef1c0f210ab4, user.id=6c34613c69674b59957862a501334be0) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:28.102993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.104535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.106555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.114322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 91/188] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:27 2022] GET /identity/v3/role_assignments?user.id=6c34613c69674b59957862a501334be0&scope.project.id=e5f5a288fd234f50b48bef1c0f210ab4&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.115630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.125878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.192166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fd5aa-cc7f-44a6-a377-0f28a83e465b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.244106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.244771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.245047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.245313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.262461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.263996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.267005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 98/189] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.267829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.273735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d208-a91b-4c45-84d2-26c77ed1cde6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.327069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.327965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.328248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.328553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.342986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:28.350524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:28.351391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:28.352341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 92/190] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:28.353004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.378959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.401912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c392657-182f-41a0-baab-922e0b233679 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.437464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.443544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.444418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.444864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.445125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.446246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.447691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.449287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 99/191] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.453465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.472363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60c139-06b8-4d0f-a55f-f54f27f45faa None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.505559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.506162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.506517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.506961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.508250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.509752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.511350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 93/192] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.512124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.549112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8832aa71-2286-418b-afd7-32887de6e027 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.583967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.584817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.585086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.585352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.607461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8de8-6ded-4841-bea2-f83b31d52822 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.608227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.640712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.641310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.641585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.641849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.642494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:28.642790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.644147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.645483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.651769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.652775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 94/193] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.653319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.653442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.748683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 100/194] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:28.749492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.755860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e494492-be4e-4f4b-b60d-20eddc8a9bc3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.788521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.789153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.789428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.789698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.799611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:28.803351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:28.805488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:28.806364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:28.807164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 95/195] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:28.807835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.848848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0152b-1c69-490b-8aba-443436db9dc5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.866828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:28.884017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.884659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.884933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.885200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.885836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.887269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.907870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 101/196] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:28 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:28.908595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:28.930706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3e09f-12df-4c6f-9b72-d860042641f6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:28.967882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:28.968577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:28.968815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:28.969063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:28.969865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=9dcd38c10c5f47bf8d47fab51d035bf9) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:28.970178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:28.971745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:28.984334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 96/197] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/role_assignments?user.id=9dcd38c10c5f47bf8d47fab51d035bf9&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:28.984334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.040494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cdec1b-a9d7-472b-a5f0-d50b9600f1cc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.054342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.091298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.092104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.092491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.092768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.094046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.096670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.100245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 102/198] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.102557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.147673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9f1c7-3b7a-4312-b458-f73a4e209db8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.174304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.199255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.200477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.201485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.201959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.203624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.205520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.207955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 97/199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:28 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.208960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.251596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2bd96-2ce2-4b10-a1b8-fd7b7db07e3e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.298072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.298654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.298926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.299199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.299863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:29.300164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.301695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.313143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 103/200] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.314106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.333455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653a323-995f-4de9-8f9d-2571662c7647 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.370469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.380337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.381196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.381640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.382074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.437250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502e6f5-6eb2-4d2a-908b-b99d73eb0f80 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.466220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.475474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.476366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.476853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.477310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.478568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=18342a993b074c229af999585ca08a2b) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:29.479083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.480853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.481799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.483670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.487694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 104/201] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/role_assignments?user.id=18342a993b074c229af999585ca08a2b&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.493049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.609493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 98/202] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:29.615616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.693541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e326c8-75c0-46ff-b00a-4ffd4ed0f6db None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.696510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.742230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.743203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.744148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.745126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] PATH_INFO: `/v3/users/0bce92399ffd4a4cb1398f51eda5dae7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.746491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] RBAC: Authorizing `identity:get_user(user_id=0bce92399ffd4a4cb1398f51eda5dae7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.748140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.750127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 105/203] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/users/0bce92399ffd4a4cb1398f51eda5dae7 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.757865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.808122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:29.837201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb20dc6-7a75-41cf-9d8f-5e0e7110fa0b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:29.904000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:29.904665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:29.904938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:29.905210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:29.908079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:29.909603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:29.911295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 99/204] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:29.921294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:29.950248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31838f-ab12-4f7d-9222-f6c7ec2a6ed1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:30.021325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:30.021945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:30.022219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:30.022488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:30.101388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b17dc6-bc82-4bd4-84a8-b51db6014f83 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:30.110837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:30.154417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:30.154936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:30.155213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:30.155483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:30.160317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=18342a993b074c229af999585ca08a2b) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:30.160317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:30.160317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:30.165788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 100/205] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/role_assignments?user.id=18342a993b074c229af999585ca08a2b&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=49615db920c64505958165b68049a95e => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:30.169484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:30.171107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:30.328769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 106/206] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:30.334480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:30.365362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:30.474454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b4796-e41a-4f47-83b7-a24f8e33483d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:30.515165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:30.516167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:30.516688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:30.517159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:30.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:30.520446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14e715-b522-41a2-b8f3-1a30079a82ea None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:30.522386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 101/207] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:30 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:30.528632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:30.641301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:30.680652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a69ac-6cdc-4690-999d-8a3afdef0d78 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:30.730229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:30.730770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:30.731045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:30.731442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:30.732814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706, user.id=0bce92399ffd4a4cb1398f51eda5dae7) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:30.733229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:30.734748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f624a-dfbf-4979-b6ba-88b57ff685ce None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:30.748818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 107/208] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:30 2022] GET /identity/v3/role_assignments?user.id=0bce92399ffd4a4cb1398f51eda5dae7&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:31.546372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7ab99-8142-4750-aab2-bb41ee12641a None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.546911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7ab99-8142-4750-aab2-bb41ee12641a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.546911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7ab99-8142-4750-aab2-bb41ee12641a None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.548382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 102/209] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:31.553635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9bd5-105f-47ee-ae3f-fc11591b510e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.554033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9bd5-105f-47ee-ae3f-fc11591b510e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.554194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9bd5-105f-47ee-ae3f-fc11591b510e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.572671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-677b9bd5-105f-47ee-ae3f-fc11591b510e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:31.573849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-677b9bd5-105f-47ee-ae3f-fc11591b510e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:31.617781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e015-6e7f-42e0-9752-3be6911c2b3e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.618097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e015-6e7f-42e0-9752-3be6911c2b3e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.618321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7e015-6e7f-42e0-9752-3be6911c2b3e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.619773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 103/210] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:31.625851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d5e-8603-469b-8a64-f93cf1803876 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.626131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d5e-8603-469b-8a64-f93cf1803876 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.626406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e699d5e-8603-469b-8a64-f93cf1803876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.643992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6e699d5e-8603-469b-8a64-f93cf1803876 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:31.645712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6e699d5e-8603-469b-8a64-f93cf1803876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:31.647151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 108/211] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:31.712886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 104/212] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:31.897196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e88ae-33e5-4e06-8861-8da4d20f230c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.897512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e88ae-33e5-4e06-8861-8da4d20f230c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.897736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e88ae-33e5-4e06-8861-8da4d20f230c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.899092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 109/213] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:31.904287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:31.904746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:31.905157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:31.922921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:31.924251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:31.927740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e88ae-33e5-4e06-8861-8da4d20f230c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:31.938294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.000038 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 105/214] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:32.000929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.018687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.061286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e88ae-33e5-4e06-8861-8da4d20f230c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.098426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.099098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.099375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.099636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.100788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.102214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.104238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 110/215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:32.109351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.117240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980848c-c6d9-4251-8bb3-cb754e06fbab None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.162669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.163216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.163488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.163758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.174434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:32.179783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:32.180685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:32.181485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 106/216] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:32.186675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.198431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.228875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65aed9c-b476-4137-b30e-3b8e5838d3ff None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.264580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.265967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.265967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.265967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.266604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.268099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.278064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.292066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 111/217] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:32 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:15:32.292066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.314274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d445a-1fc6-4802-927b-f1206a67bbfc None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.368216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.369328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.369802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.370350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.372040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:32.372629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.375009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.389651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 107/218] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:32.395542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.431780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf435a-3b7f-4124-b146-a6daaba69225 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.472380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.473667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.474152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.474650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.492479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:32.494162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:32.498195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:32.499087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:32.499854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 112/219] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:32.508586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.545218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395de7-216c-4f05-bf69-a975f5e9aeae None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.572482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.593261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.594952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.595601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.596082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] PATH_INFO: `/v3/users/db0e17bcb19d482f85ec24de0435d287` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.618975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] RBAC: Authorizing `identity:get_user(user_id=db0e17bcb19d482f85ec24de0435d287)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.621388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.627586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 108/220] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/users/db0e17bcb19d482f85ec24de0435d287 => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:32.629836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.639662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c47f54-ccb0-436e-bb31-24bd9341d700 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.681352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.682333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.682783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.683237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.684098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:32.684599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.686357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.696190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 113/221] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:32.701859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:32.763571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cafd8-3158-4fc3-b7e8-7487092de4c1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.795917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.805567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.806546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.807052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.807518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.839547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fb414-ffea-47a8-bc34-abad478f3f35 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:32.855938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:32.887774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.889797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2139e99-323d-4e2b-9891-97e8f2cab619 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.890140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:32.890789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.891058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.891325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.892496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:32.895322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d84a9-c8c4-4a1c-becb-db331b7d5565 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:32.897389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 114/222] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:32.900020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0199d-7b47-4923-8790-72a0749b676c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:32.900574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0199d-7b47-4923-8790-72a0749b676c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:32.900936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0199d-7b47-4923-8790-72a0749b676c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:32.902310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 115/223] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:32.903013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0199d-7b47-4923-8790-72a0749b676c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.005233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 109/224] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:15:33.006916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.007456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.007916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.015857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0199d-7b47-4923-8790-72a0749b676c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.030478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:33.032032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:33.052410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.053044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.053319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.053589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.054791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:33.056362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:33.057991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 116/225] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:32 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:33.058742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.062374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:33.116705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 110/226] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:33.117540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.164652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:33.224284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586ad8e-5b2f-4b54-806e-9dba412cb1b9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.273527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.274200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.274481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.274756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.277594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:33.279101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:33.280803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d3770d-5629-447f-a1d7-cdb8b1bf15b9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.281170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 117/227] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:33 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:33.287687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.318798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.319835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.320323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.320808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/18342a993b074c229af999585ca08a2b/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.323468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, user_id=18342a993b074c229af999585ca08a2b, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:33.325195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:33.351242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 111/228] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:33 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/18342a993b074c229af999585ca08a2b/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:33.435546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79bde7-f059-4749-9b22-55ca54d0094a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.452523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.453598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:33.473738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.474359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.474636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.474907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.475627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=db0e17bcb19d482f85ec24de0435d287) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:33.475921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:33.477425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:33.483257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:33.488440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 118/229] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:33 2022] GET /identity/v3/role_assignments?user.id=db0e17bcb19d482f85ec24de0435d287&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:33.607275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e4584-e4ae-40b5-bbb1-ee1483ad63d2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.650577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.651234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.651512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.651779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.663266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:33.668329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:33.669260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:33.670244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 112/230] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:33.678187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.788098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe89c48-b839-455a-ad08-d1cdc4bab91e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:33.836595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:33.837261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:33.837536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:33.837852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:33.838511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:33.838811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:33.840181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:33.850223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 119/231] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:33.856327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:33.965794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:33.995986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013ab2-fbda-4665-bd04-38549a656c40 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:34.030292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:34.030920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:34.031192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:34.031460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:34.032606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:34.034031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:34.036848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 113/232] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:33 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:34.042320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:34.112736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:34.192158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcafd2-f2e3-4084-bfe2-4180466b86f5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:34.239020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:34.239646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:34.239918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:34.240188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:34.244962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:34.246405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615023d4-5d3f-402c-8407-50bc9ed03480 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:34.248343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 120/233] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:34 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:34.254095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:34.405619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6712451-a49d-475d-92ca-824b7381379a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:34.453973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:34.454573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:34.454856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:34.455132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/18342a993b074c229af999585ca08a2b/roles/49615db920c64505958165b68049a95e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:34.457883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, user_id=18342a993b074c229af999585ca08a2b, role_id=49615db920c64505958165b68049a95e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:34.459324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90feac-fa1f-442d-94b8-50b98552c697 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:34.481507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 114/234] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:34 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/18342a993b074c229af999585ca08a2b/roles/49615db920c64505958165b68049a95e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:35.066962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c502e2-1710-4007-ad49-3fb10fb4b1f7 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.067956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c502e2-1710-4007-ad49-3fb10fb4b1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.068496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c502e2-1710-4007-ad49-3fb10fb4b1f7 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.070016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 121/235] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:35.075736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463acc0-6499-412f-b64b-3a10ee573b18 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.076269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463acc0-6499-412f-b64b-3a10ee573b18 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.076269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463acc0-6499-412f-b64b-3a10ee573b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.095624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:35.101539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c463acc0-6499-412f-b64b-3a10ee573b18 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:35.102743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c463acc0-6499-412f-b64b-3a10ee573b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:35.196413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 115/236] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:35.200838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ded7a-4d41-463f-be69-13655dfbd853 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.201336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ded7a-4d41-463f-be69-13655dfbd853 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.201797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ded7a-4d41-463f-be69-13655dfbd853 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.203297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 122/237] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:35.209659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.209942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.210212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.251033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:35.252226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:35.320424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 116/238] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:35.585519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ded7a-4d41-463f-be69-13655dfbd853 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:35.589422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:35.708653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:35.725927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ded7a-4d41-463f-be69-13655dfbd853 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:35.744328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:35.744492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168a5c6-1aff-4bef-a8d0-2d45a93d7b98 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:35.779836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:35.780673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.780985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.781289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.791779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:35.792264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:35.792470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:35.792713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:35.792978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:35.794071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:35.795508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:35.797058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:35.797935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:35.798728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 117/239] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:35.802127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 123/240] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:35.809764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:35.811673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:35.986557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b285026-cb41-41af-a3fa-33fe2792ddc3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.010530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2a07b9-23db-4934-90bd-6d7f3f95b81a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.044661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:36.052313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.053198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.053474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.053740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.054557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:36.060730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5b53c-db1d-4cf0-a6da-3d98f7bd2c4a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:36.074367 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.074977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.075252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.075518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.076165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:36.076489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:36.077856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:36.092026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 124/241] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:36.102463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:36.107871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 118/242] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:35 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:36.167996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:36.251824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cca59b-eb3b-45d2-8798-8d4a5cff29e9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.270528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d043a-0f60-4262-9af3-8c7af701c703 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.270830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d043a-0f60-4262-9af3-8c7af701c703 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.271060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d043a-0f60-4262-9af3-8c7af701c703 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.272668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 119/243] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:36.277768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417e549-d144-4e99-bff3-7cf428fca193 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.278069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417e549-d144-4e99-bff3-7cf428fca193 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.278351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0417e549-d144-4e99-bff3-7cf428fca193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.295077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.295716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.295992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.296266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] PATH_INFO: `/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.308155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0417e549-d144-4e99-bff3-7cf428fca193 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:36.309417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0417e549-d144-4e99-bff3-7cf428fca193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:36.324175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] RBAC: Authorizing `identity:get_user(user_id=f7f0b8c62fa54349a8a6865d478ae4d1)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:36.325977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:36.327914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 125/244] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1 => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:36.334111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:36.401140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 120/245] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:36.402570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163c23-e862-4c9f-be32-55a251f9920b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.404005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163c23-e862-4c9f-be32-55a251f9920b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.404385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86163c23-e862-4c9f-be32-55a251f9920b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.408422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 121/246] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:36.411128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.411405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.411671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.432773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:36.433876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:36.470334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:36.474885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783d4-f546-447f-a42c-db26ba766284 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.512932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.513639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.514107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 122/247] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:36.514214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.514381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.589684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:36.591199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:36.698087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:36.703148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 126/248] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:36.709466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:36.770152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:36.778436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:36.847232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d062b5-1096-4c0e-a171-f17f579715a9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.890578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.891162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.891436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.891784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.893027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:36.895033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:36.898115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 123/249] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:36.902559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975c88-0cfe-4206-ab8c-3b09102c284d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:36.904396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:36.944192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:36.944853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:36.945123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:36.945549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:36.958839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:36.964755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:36.965662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:36.967121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 127/250] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:36.967981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.027617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af8879-2f62-4189-9076-2b9a92a557a3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.039478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.067139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.067742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.068016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.068282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.069029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=f7f0b8c62fa54349a8a6865d478ae4d1) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:37.069331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:37.070765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:37.077715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 124/251] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/role_assignments?user.id=f7f0b8c62fa54349a8a6865d478ae4d1&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:37.078871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.122794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.238059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916e58c-8a19-4b85-b60d-4e3d9f7fa9fc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.271763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847cd82-478a-4393-a6d2-89e4ec8a3b96 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.324438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.325099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.325378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.325648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.338710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.339048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:37.339336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.339611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.339878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.340582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:37.340877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:37.342264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982229d-8a4d-4561-854c-24d2471acff3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-dd613ff3-ce50-4ead-b452-13e6d6945cab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:37.345502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:37.346407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:37.347236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 128/252] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:37.350887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa2eb-da88-4efa-8700-93f8b3fe33df None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.350887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa2eb-da88-4efa-8700-93f8b3fe33df None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.350887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaa2eb-da88-4efa-8700-93f8b3fe33df None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.350887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 129/253] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:37.352163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 125/254] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:37.356195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa2eb-da88-4efa-8700-93f8b3fe33df None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.358020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.358302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.358573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.380987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:37.382459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:37.420473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.494802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaa2eb-da88-4efa-8700-93f8b3fe33df None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.505964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 126/255] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:37.506844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.520003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.551053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.551661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.551934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.552198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.552941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:37.553604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:37.554988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:37.572819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 130/256] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:37.578910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.688747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce1df6-d795-4007-96e2-e9db434ae6e0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.745828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.746505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.746780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.747045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] PATH_INFO: `/v3/users/db0e17bcb19d482f85ec24de0435d287` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.748188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] RBAC: Authorizing `identity:get_user(user_id=db0e17bcb19d482f85ec24de0435d287)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:37.749780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:37.756381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 131/257] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/users/db0e17bcb19d482f85ec24de0435d287 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:37.759758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f26561-4fcf-40b1-89cc-b55e509c4eaf None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.761888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.808155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.808815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.809090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.809360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:37.810524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:37.812116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:37.813804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 127/258] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:37.822852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:37.865358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.876539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:37.950253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57690594-2afe-4a6d-8cfc-5576d9462c74 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.978382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba364fcc-3f4a-4e98-a185-029396181618 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:37.987011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:37.987618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:37.987928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:37.988195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.054525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.055055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.055317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.055581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.061059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.062152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.062545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.063626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.064686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 132/259] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:38.065531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.098965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:38.190883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 128/260] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:15:38.192249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.199206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69cb4e-f3a1-4c21-93e8-92bdc911aa19 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.218053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:38.235222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.235951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.236231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.236521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.263474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:38.272631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:38.274865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:38.276896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:38.278361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 133/261] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:38.280331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.360106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4484a5-ced2-4c9e-91fa-209bcbf01443 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.394598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.395241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.395569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.395845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/db0e17bcb19d482f85ec24de0435d287/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.397742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, user_id=db0e17bcb19d482f85ec24de0435d287, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.399164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.401261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:38.414479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 129/262] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:38 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/db0e17bcb19d482f85ec24de0435d287/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:38.415665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.508160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822277e5-f8a9-4b06-9bcc-ae3da0bd2e13 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.538574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131e822-fb4e-47f6-8a53-632b8c487a44 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.577777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.578785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.579265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.579729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.580665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:38.581166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.582969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.593151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 130/263] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:38.600188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.600829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.601108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.601383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.602266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.602588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.605575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.605970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 134/264] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:38.611923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.725643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:38.729291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:38.806953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31f548-6c5e-4c2c-a897-e66e8f66ebae None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.832767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250c2f9-e388-4d15-9f14-e6373e33d2e3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:38.870002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.870576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.870856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.871135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] PATH_INFO: `/v3/users/18342a993b074c229af999585ca08a2b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.872323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] RBAC: Authorizing `identity:get_user(user_id=18342a993b074c229af999585ca08a2b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.879762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.881467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 131/265] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/users/18342a993b074c229af999585ca08a2b => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:38.890622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:38.910320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:38.910930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:38.911203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:38.911469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:38.912190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=18342a993b074c229af999585ca08a2b) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:38.912516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:38.913955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:38.929019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 135/266] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/role_assignments?user.id=18342a993b074c229af999585ca08a2b&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:39.087607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2ecd98-d6b9-4075-a79d-9db99b90b6df None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:39.128604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:39.141779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:39.142396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.142670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.142936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.244821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:39.246310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacbabde-d04b-416f-af83-3b49bf555989 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:39.365517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 132/267] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:39.370747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:39.486512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b21a7-e9ba-4362-b80c-6d413ce8d1ca None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.486796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b21a7-e9ba-4362-b80c-6d413ce8d1ca None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.486970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b21a7-e9ba-4362-b80c-6d413ce8d1ca None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.488813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 133/268] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:39.493818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313213d-84d7-4233-b96e-07e2e6be3375 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.494096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313213d-84d7-4233-b96e-07e2e6be3375 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.494360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313213d-84d7-4233-b96e-07e2e6be3375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.518735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9aaab9-784c-43fd-9c52-a79ef25f5266 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:39.535475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2313213d-84d7-4233-b96e-07e2e6be3375 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:39.537020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2313213d-84d7-4233-b96e-07e2e6be3375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:39.548652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:39.573851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:39.594815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:39.595460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.595736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.596009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.597711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:39.599722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:39.601884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 136/269] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:39 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:39.611829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:39.671548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 134/270] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:39.755943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb901f3-a775-418f-8ac5-b46db4e5eaea None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:39.804001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:39.804648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.804924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.805192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.805916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=18342a993b074c229af999585ca08a2b) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:39.806221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:39.807676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfdbc93-feff-4250-91eb-264fbace95f7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:39.811836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674de42-bbcf-4b96-8c3d-a83668c148ad None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.812129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674de42-bbcf-4b96-8c3d-a83668c148ad None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.812453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7674de42-bbcf-4b96-8c3d-a83668c148ad None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.814111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 135/271] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:39.814469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 137/272] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:39 2022] GET /identity/v3/role_assignments?user.id=18342a993b074c229af999585ca08a2b&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=49615db920c64505958165b68049a95e => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:39.830985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862223-d95e-4f62-b450-8a234042bfeb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:39.831337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862223-d95e-4f62-b450-8a234042bfeb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:39.838662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862223-d95e-4f62-b450-8a234042bfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:39.876389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c2862223-d95e-4f62-b450-8a234042bfeb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:39.877585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c2862223-d95e-4f62-b450-8a234042bfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:39.958928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 136/273] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:40.079025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097abc-f3a2-4e0d-ba05-1e295921a5ca None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.079330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097abc-f3a2-4e0d-ba05-1e295921a5ca None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.079580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7097abc-f3a2-4e0d-ba05-1e295921a5ca None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.081218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 138/274] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:40.086124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.086458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.086746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.109793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7097abc-f3a2-4e0d-ba05-1e295921a5ca None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.109971 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:40.110981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:40.155786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.191186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 137/275] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:40.216402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7097abc-f3a2-4e0d-ba05-1e295921a5ca None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.251663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.252313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.252598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.252910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.263458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:40.268961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:40.269807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:40.271057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 139/276] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:40.276484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.294041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.299438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.406785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eb350-6e33-423d-8111-78c6a88b0db6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.412736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.416981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d964c-fa95-4447-9464-fbe61d1a73ac None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.443658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.444389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.444668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.444937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.445691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:40.445991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.447355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.453540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.454267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.454614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.454953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.457014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 138/277] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:40.462658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.469000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:40.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:40.475914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:40.476971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 140/278] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:40.481791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.558300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.585065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa386aab-90b9-434d-883e-ed1b579e5928 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.585214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be268c6-cb77-4ad0-bff0-b06618232d8a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.618550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.619171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.619440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.620804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.622245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.623776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 141/279] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:40.628272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.630597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.632007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.632288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.632591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] PATH_INFO: `/v3/users/d1a844e566a24ef09e7ad9b54ced1954` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.650615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] RBAC: Authorizing `identity:get_user(user_id=d1a844e566a24ef09e7ad9b54ced1954)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.652074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.658380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 139/280] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/users/d1a844e566a24ef09e7ad9b54ced1954 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:40.658380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.659114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.730120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf11b2-fe54-4dc9-9517-5c90cc45a83c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.763328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.763926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.764200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.764503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.765129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:40.765433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.766539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5456b752-2d0d-4fd6-a7ee-40569628397a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.766817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.780044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 142/281] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:40.780044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.808245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.808907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.809428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.809558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.810095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.811522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.820257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.835777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 140/282] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:15:40 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:40.835777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:40.912920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:40.924113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078feb88-c318-4381-a7e1-2297646c18d5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.955712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b113019-c642-4965-9aa3-ba757e74a3fe None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:40.974462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.975524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:40.975808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:40.976079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:40.976845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, user.id=d1a844e566a24ef09e7ad9b54ced1954, scope.system=all) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:40.977148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:40.979530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:40.987768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 143/283] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/role_assignments?user.id=d1a844e566a24ef09e7ad9b54ced1954&scope.system=all&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:40.999281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:40.999981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.000263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.000595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] PATH_INFO: `/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.002118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] RBAC: Authorizing `identity:get_user(user_id=f7f0b8c62fa54349a8a6865d478ae4d1)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:41.004230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:41.006420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 141/284] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:40 2022] GET /identity/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:41.012590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:41.128007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b61337-5877-4600-8161-751b456788a8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:41.174014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:41.174598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.174872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.175676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.177656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:41.179140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d57cd65-933b-478b-b45d-aba614dfeb08 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:41.181073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 144/285] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:41 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:41.187121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:41.257167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:41.308556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913b6c88-e8d5-4aa8-a1ad-d03e5857c905 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:41.344898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:41.345562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.345838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.346107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/f7f0b8c62fa54349a8a6865d478ae4d1/roles/543bbe978c784ef38f0c36d95b5a194b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.348126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, user_id=f7f0b8c62fa54349a8a6865d478ae4d1, role_id=543bbe978c784ef38f0c36d95b5a194b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:41.349616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f4386f-c80e-4a8b-957e-23e0fd0b387b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:41.369560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 142/286] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:15:41 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/users/f7f0b8c62fa54349a8a6865d478ae4d1/roles/543bbe978c784ef38f0c36d95b5a194b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:41.373521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365214b-3f85-4569-8e7d-885b0ebdc05c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.373801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365214b-3f85-4569-8e7d-885b0ebdc05c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.374076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365214b-3f85-4569-8e7d-885b0ebdc05c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.375598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 145/287] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:41.381884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.382191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.382431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.402243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:41.403479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:41.480682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 143/288] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:41.782175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365214b-3f85-4569-8e7d-885b0ebdc05c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:41.815412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:41.919798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365214b-3f85-4569-8e7d-885b0ebdc05c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:41.962320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:41.962949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:41.963220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:41.963483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:41.978832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:41.984336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:41.985297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:41.986732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 146/289] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:41.993389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:42.058225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:42.134121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92c6a7-4887-46c5-8ffa-d7ec09190b05 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:42.181005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:42.181608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:42.181879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:42.182142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:42.182794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:42.183085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:42.184472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:42.195496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 144/290] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:42.201259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:42.320105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:42.347231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494ae7-0ee9-419a-b284-5525077e3d81 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:42.403354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:42.403984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:42.404253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:42.404580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] PATH_INFO: `/v3/users/db0e17bcb19d482f85ec24de0435d287` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:42.405722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] RBAC: Authorizing `identity:get_user(user_id=db0e17bcb19d482f85ec24de0435d287)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:42.407174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:42.409044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 147/291] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:42 2022] GET /identity/v3/users/db0e17bcb19d482f85ec24de0435d287 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:42.414571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:42.567472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fde46-5972-46ec-9628-3bcbaadef6fa None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:42.583857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:42.618352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:42.618968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:42.619247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:42.619519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:42.711122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:42.712960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:42.849092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 145/292] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:42.854940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:43.000964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105545d5-d3fd-4a5c-a38d-0e3ec05d8a63 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:43.051900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:43.052567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.052847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.053118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.054241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:43.055674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462c352-252f-4d52-8bac-d710b82c7a1a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:43.057533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 148/293] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:42 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:43.063183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:43.083024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:43.220055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301c4cd-a0c6-4958-8d70-aefcc9710bfb None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:43.267201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:43.267813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.268093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.268387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.269140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=db0e17bcb19d482f85ec24de0435d287) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:43.269446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:43.270870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531e117-4263-40eb-8c7b-54b8b21669fa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:43.281010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 146/294] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:43 2022] GET /identity/v3/role_assignments?user.id=db0e17bcb19d482f85ec24de0435d287&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:43.605763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ed71-d952-406f-a3b3-599fbf6f2508 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.606461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ed71-d952-406f-a3b3-599fbf6f2508 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.606935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ed71-d952-406f-a3b3-599fbf6f2508 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.608499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 149/295] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:43.614538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9b48-83d7-40c9-be8b-08c2a1232942 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.615441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9b48-83d7-40c9-be8b-08c2a1232942 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.615908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d9b48-83d7-40c9-be8b-08c2a1232942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.639570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a32d9b48-83d7-40c9-be8b-08c2a1232942 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:43.641091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a32d9b48-83d7-40c9-be8b-08c2a1232942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:43.724419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 147/296] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:43.778814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119a18-e83d-468e-9639-6eccccfffcb1 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.779420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119a18-e83d-468e-9639-6eccccfffcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.779887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b119a18-e83d-468e-9639-6eccccfffcb1 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.781857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 150/297] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:43.787321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14329683-9b7e-4956-b5da-635e8280be9f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:43.787602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14329683-9b7e-4956-b5da-635e8280be9f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:43.787921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14329683-9b7e-4956-b5da-635e8280be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:43.796756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:43.809198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-14329683-9b7e-4956-b5da-635e8280be9f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:43.810383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-14329683-9b7e-4956-b5da-635e8280be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:43.885028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 148/298] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:44.012190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b119a18-e83d-468e-9639-6eccccfffcb1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.021751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d64e9-0238-4af8-b42e-a5eefc03307d None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.022071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d64e9-0238-4af8-b42e-a5eefc03307d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.022300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d64e9-0238-4af8-b42e-a5eefc03307d None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.023802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 149/299] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:44.028796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4522c0d-c647-4742-b946-8ab4a6627773 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.029080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4522c0d-c647-4742-b946-8ab4a6627773 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.029370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4522c0d-c647-4742-b946-8ab4a6627773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.036011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.052164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f4522c0d-c647-4742-b946-8ab4a6627773 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:44.053965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f4522c0d-c647-4742-b946-8ab4a6627773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:44.132640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b119a18-e83d-468e-9639-6eccccfffcb1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.136368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 150/300] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:44.169349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.170507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.171000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.171454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.172370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405b21-f6a5-40f9-b060-4757773dd16b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.172659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405b21-f6a5-40f9-b060-4757773dd16b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.172931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb405b21-f6a5-40f9-b060-4757773dd16b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.174387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 151/301] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:44.179079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.179365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.179641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.187531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:44.192897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:44.193790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:44.194781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 151/302] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:44.195415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.199927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:44.201268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:44.261629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.272399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 152/303] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:44.273115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.284852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.313724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b30a7a-bc7f-4069-93a0-04ee24bd44a6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.357429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.358412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.359313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.359777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.373293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:44.381342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:44.382193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:44.383029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:44.385546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 152/304] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:44.388986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a70f4c-2186-4153-93a8-3f4d5b843b95 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.391356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.430866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.431468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.431746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.432021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.432710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:44.433011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.434360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.442783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 153/305] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:44.443568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.498297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb108e-c3f9-403e-a10e-a202ba57bf02 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.502582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.532839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.533436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.533710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.533974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.534644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:44.534943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.536327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.541786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.545746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 153/306] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:44.546855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.555440 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41785687-1813-4cbe-8cdc-2bbadb605c3f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.588536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.589153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.589432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.589703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.590791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.592210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.593788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 154/307] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:44.594540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.644563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e6a6e-9382-4482-9e5b-47f8be7d2a1a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.676598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.677213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.677535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.677807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] PATH_INFO: `/v3/users/d1a844e566a24ef09e7ad9b54ced1954` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.678900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] RBAC: Authorizing `identity:get_user(user_id=d1a844e566a24ef09e7ad9b54ced1954)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.680358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.681924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 154/308] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/users/d1a844e566a24ef09e7ad9b54ced1954 => generated 293 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:44.682663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.693300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061eeb0-0b22-474d-8378-619147629511 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.716616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.727413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.728279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.728575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.728842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.738433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:44.743584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:44.744474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:44.745272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 155/309] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:44.746414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.750171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.781266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157db928-a382-48a5-b4c3-83c5d708865f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.813644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.814265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.814537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.814804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] PATH_INFO: `/v3/users/937f24f2314e4f19b6bbfe53d206e76c` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.827764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] RBAC: Authorizing `identity:get_user(user_id=937f24f2314e4f19b6bbfe53d206e76c)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.829243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.830830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 155/310] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/users/937f24f2314e4f19b6bbfe53d206e76c => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:44.832160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.842520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07555-ee8a-4e99-b9d7-bb4a9d626020 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.881812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.882417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.882692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.882956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] PATH_INFO: `/v3/groups` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.883569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.885033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.902007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 156/311] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Sep 22 12:15:44 2022] POST /identity/v3/groups => generated 229 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:15:44.902746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:44.937997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86594b-61f7-4394-9f8f-df6549190518 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:44.971074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.972781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:44.981974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:44.982655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:44.982933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:44.983202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] PATH_INFO: `/v3/system/users/d1a844e566a24ef09e7ad9b54ced1954/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:44.984775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d1a844e566a24ef09e7ad9b54ced1954, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:44.986145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:44.996553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 156/312] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Sep 22 12:15:44 2022] PUT /identity/v3/system/users/d1a844e566a24ef09e7ad9b54ced1954/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:45.004587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:45.053080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e1f42-a3ed-48a9-afb1-844f83116644 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.097045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.097775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.098058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.098328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.100003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:45.100438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.101949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.113829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 157/313] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:45.121094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:45.137763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a890d6e-d2fb-49f5-9cbc-5d6f702f21c4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.184996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.185537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.185814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.186085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.187015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, user.id=937f24f2314e4f19b6bbfe53d206e76c, scope.system=all) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:45.187324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.189381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.197647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 157/314] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 22 12:15:44 2022] GET /identity/v3/role_assignments?user.id=937f24f2314e4f19b6bbfe53d206e76c&scope.system=all&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:45.261450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa794a-f1da-45f8-949f-0971b365c4f5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.271158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:45.303061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.303719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.303997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.304271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] PATH_INFO: `/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.305486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=f7f0b8c62fa54349a8a6865d478ae4d1)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.306944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.308950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 158/315] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:45 2022] GET /identity/v3/users/f7f0b8c62fa54349a8a6865d478ae4d1 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:45.314288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:45.379470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:45.428720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f333e5c-65c6-48f5-880d-f748230b4a0f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.463029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.463557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.463833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.464102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.534974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.536515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.591359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:45.638174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 158/316] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:45.644465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:45.747126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690601fa-3e19-4152-8802-4c694f755fa4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.781294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.782291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.782759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.783219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.785202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.786821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12524855-2b4a-440b-b089-4a07f206dd7a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.788989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 159/317] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:45 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:45.796324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:45.901438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd3625-dbbc-4a86-bcd8-4c11bb3d4cf5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:45.936758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:45.937366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:45.937638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:45.937906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:45.938623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, scope.project.id=2ca946a65b244ca2ab4c18a566425fce, user.id=f7f0b8c62fa54349a8a6865d478ae4d1) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:45.938925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:45.940405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c613ca-5cb8-47f9-9254-2b1e69b6a40a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:45.947051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 159/318] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:15:45 2022] GET /identity/v3/role_assignments?user.id=f7f0b8c62fa54349a8a6865d478ae4d1&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:47.164701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c986-a2b5-4bb2-9187-170944243e43 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.165224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c986-a2b5-4bb2-9187-170944243e43 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.165224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9c986-a2b5-4bb2-9187-170944243e43 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.166615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 160/319] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:47.171675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7dafb-881f-4dd1-ace6-8b6ceadd2641 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.172047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7dafb-881f-4dd1-ace6-8b6ceadd2641 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.172209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7dafb-881f-4dd1-ace6-8b6ceadd2641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.190016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5fc7dafb-881f-4dd1-ace6-8b6ceadd2641 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:47.191241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5fc7dafb-881f-4dd1-ace6-8b6ceadd2641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:47.206969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:47.266208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 160/320] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:47.320253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30998e-169b-4f38-aa37-22b86dad6f54 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.320578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30998e-169b-4f38-aa37-22b86dad6f54 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.320840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30998e-169b-4f38-aa37-22b86dad6f54 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.322302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 161/321] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:47.327239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.327521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.327791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.349687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:47.350867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:47.381192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c8e5-b550-4efa-a532-02a86556f6a9 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.381485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c8e5-b550-4efa-a532-02a86556f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.381742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8c8e5-b550-4efa-a532-02a86556f6a9 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.383203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 162/322] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:15:47.387806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.388090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.388399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.407838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:47.410092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:47.424359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 161/323] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:47.445799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:47.478321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 163/324] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:47.536406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:47.611866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:47.634252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338f58-396d-433a-863d-3ccb48a53489 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:47.668009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:47.668651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.668921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.669183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.670254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:47.671682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:47.673385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 162/325] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:15:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:47.678879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:47.697896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:47.776755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f04cfd-82f5-4055-8a84-3ba5327e94cc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:47.797054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e276f6d-78f6-4618-aef5-6142324330ce None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:47.808818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:47.809414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.809687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.809951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] PATH_INFO: `/v3/groups` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.810569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:47.811976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:47.827583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 164/326] 10.222.0.33 () {60 vars in 1134 bytes} [Thu Sep 22 12:15:47 2022] POST /identity/v3/groups => generated 226 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:15:47.828392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:47.833129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:47.833129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.833445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.833445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:47.852852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:47.857883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:47.858902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:47.859694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 163/327] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:47.865102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:47.872562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:47.875366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:47.955765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e95dce9-5f5c-42bb-85e0-38a9d817c3e2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:47.972698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7291b-18f0-4532-b157-128c7be90646 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:47.995093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:47.996030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:47.996530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:47.997073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.010477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:48.010639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.011184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.011463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.011727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.012414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:48.012714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.014072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:48.015523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:48.016415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:48.017225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 165/328] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:48.022558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:48.029274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 164/329] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:48.036770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:48.122879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:48.132434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:48.140458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81c8e5-b4e4-4302-8a0a-368a3ff388a6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:48.168449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0996-3ef0-489e-bc23-9079d02d67a4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:48.185187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.186379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.186649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.186920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.187578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:48.187874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.190874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.204237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 166/330] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:48.209742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:48.215004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.215643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.215918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.216184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] PATH_INFO: `/v3/users/d1a844e566a24ef09e7ad9b54ced1954` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.217349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] RBAC: Authorizing `identity:get_user(user_id=d1a844e566a24ef09e7ad9b54ced1954)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.218782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.220552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 165/331] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:48 2022] GET /identity/v3/users/d1a844e566a24ef09e7ad9b54ced1954 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:48.226123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:48.340708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b95de-1d90-4667-8ff4-c0d7e50408cc None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:48.365721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7009d-ffef-4038-ba04-f90e0be6a0ed None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:48.386884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.387551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.387829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.388100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] PATH_INFO: `/v3/users/937f24f2314e4f19b6bbfe53d206e76c` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.390101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] RBAC: Authorizing `identity:get_user(user_id=937f24f2314e4f19b6bbfe53d206e76c)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.391545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.393583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 167/332] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:48 2022] GET /identity/v3/users/937f24f2314e4f19b6bbfe53d206e76c => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:48.398753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:48.419072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.420372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.420852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.421451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.422796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] RBAC: Adding query filter params (role.id=0568cdc72a2e49fb931bbf5cf5be87e5, user.id=d1a844e566a24ef09e7ad9b54ced1954, scope.system=all) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:48.423301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.425567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f575d15-16ec-4b36-ba38-686b0c5c48de None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.430611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:48.431181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:48.443717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 166/333] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 22 12:15:48 2022] GET /identity/v3/role_assignments?user.id=d1a844e566a24ef09e7ad9b54ced1954&scope.system=all&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 510 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:48.533727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5c2c9-02e7-4fc2-bf84-8861776bce90 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:48.573910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:48.574577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:48.574852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:48.575124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] PATH_INFO: `/v3/system/users/937f24f2314e4f19b6bbfe53d206e76c/roles/543bbe978c784ef38f0c36d95b5a194b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:48.576620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=937f24f2314e4f19b6bbfe53d206e76c, role_id=543bbe978c784ef38f0c36d95b5a194b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:48.578088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1850438-e4b5-4a84-a6a7-dc64d152ebff None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:48.592094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 168/334] 10.222.0.33 () {58 vars in 1334 bytes} [Thu Sep 22 12:15:48 2022] PUT /identity/v3/system/users/937f24f2314e4f19b6bbfe53d206e76c/roles/543bbe978c784ef38f0c36d95b5a194b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:15:50.506175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b120-d674-4821-bae4-715aa3787c03 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:50.506682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b120-d674-4821-bae4-715aa3787c03 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:50.506773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a3b120-d674-4821-bae4-715aa3787c03 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:50.508186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 167/335] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:50.513808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0726a9-5167-45ff-9fa1-95436d454663 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:50.514241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0726a9-5167-45ff-9fa1-95436d454663 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:50.514411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0726a9-5167-45ff-9fa1-95436d454663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:50.541432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ee0726a9-5167-45ff-9fa1-95436d454663 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:50.542663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ee0726a9-5167-45ff-9fa1-95436d454663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:50.642963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 169/336] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:15:50.748662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b46f3-11f1-4d75-ab7d-c6c97af61e01 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:50.748979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b46f3-11f1-4d75-ab7d-c6c97af61e01 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:50.749208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b46f3-11f1-4d75-ab7d-c6c97af61e01 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:50.750520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 168/337] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:50.757550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:50.757767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:50.758037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:50.782540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:50.783745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:50.795090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:50.855653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 170/338] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:50.966540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b46f3-11f1-4d75-ab7d-c6c97af61e01 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.092989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b46f3-11f1-4d75-ab7d-c6c97af61e01 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.135245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.135878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.136149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.136440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.153520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:51.158315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:51.159240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:51.160026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 169/339] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:51.167234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.287059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.298451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc2685-ef1e-4e39-802c-77047a93a6db None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.355551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.356585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.357180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.357647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.358510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:51.359005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:51.360870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:51.375447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:51.379576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 171/340] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:51.386411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.402419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:51.440878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7457c4e-a571-47b8-bd19-4af7b8719aad None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.480483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.481233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.481511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.481780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.495673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-7f963d9a-382d-4836-85de-f0c3bcbf11f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:51.501506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:51.502507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:51.503315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 170/341] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:51.506429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0232-e5d1-41ab-9329-96a674812e75 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.506429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0232-e5d1-41ab-9329-96a674812e75 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.506429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0232-e5d1-41ab-9329-96a674812e75 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.507188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 171/342] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:51.513698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.514151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.514648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.538691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:51.540022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:51.616505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 172/343] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:51.617529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.619055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb83ba8-17a9-4532-9084-f83d80cf6764 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.679235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:51.704615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.706398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.706897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.707387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.729279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e29c5-d12d-4d3d-8ccc-d58511e169a0 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.765051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.765878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.766042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.766287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.766948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:51.767246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:51.769170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85181e27-40a3-409d-8ada-a95e57b9f46f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:51.777739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 173/344] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:51.778976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19da7-fd46-4aae-8465-a326eb7749cd None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.779261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19da7-fd46-4aae-8465-a326eb7749cd None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.779532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b19da7-fd46-4aae-8465-a326eb7749cd None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.781963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 174/345] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:51.785623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b19da7-fd46-4aae-8465-a326eb7749cd None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:51.788774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:51.790225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc24a2-35db-4906-8458-870abe2ada38 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:51.803221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:51.951933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b19da7-fd46-4aae-8465-a326eb7749cd None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:51.963311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 172/346] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:51.963756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:51.964676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.964959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.965231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:51.989048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:51.990163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:51.990163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:51.990435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.009464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:52.010659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:52.105056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 173/347] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:52.105742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.109583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:52.111068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:52.156770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.227921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31feb0b7-ef38-46d0-bc8d-a8774e75d0ba None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.272240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad0014-0873-42b9-932f-444214486463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.272916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad0014-0873-42b9-932f-444214486463 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.273190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad0014-0873-42b9-932f-444214486463 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.273462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad0014-0873-42b9-932f-444214486463 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.274579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad0014-0873-42b9-932f-444214486463 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:52.276166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad0014-0873-42b9-932f-444214486463 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:52.278527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 174/348] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:52.279783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad0014-0873-42b9-932f-444214486463 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.349513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 175/349] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:52.350376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.365265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.490661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad0014-0873-42b9-932f-444214486463 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.499326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.536609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32333b68-c021-4624-9037-fa97b6f58ec2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.545298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.546463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.546463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.546844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.563091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:52.568147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:52.569309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:52.570337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:52.571941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 175/350] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:52.573711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.590350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.591302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.591575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.591842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.600414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:52.601883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:52.604566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 176/351] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:52.605576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.695854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.699623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5db4-29c6-48b6-a5df-42a582d15bdf None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.734927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ba08a-9cca-4ced-84a2-1ef36367f7d1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.746368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.747572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.748060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.748652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.750027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:52.751736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b5949b-db38-4df2-8fed-6b0e9fc4005c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:52.754142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 176/352] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:52.755617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062c51-b7ca-4a13-a09e-0acfdc80bde0 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.755898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062c51-b7ca-4a13-a09e-0acfdc80bde0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.756163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40062c51-b7ca-4a13-a09e-0acfdc80bde0 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.757497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 177/353] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:52.760554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40062c51-b7ca-4a13-a09e-0acfdc80bde0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.788697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.790218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.790787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.792229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.804393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.806649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:52.811890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:52.813017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:52.813824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 177/354] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:52.814488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.890642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40062c51-b7ca-4a13-a09e-0acfdc80bde0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.930572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.931213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.931488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:52.931753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:52.932793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] RBAC: Adding query filter params (name=member) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:52.932793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:52.934113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:52.945282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 178/355] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:52.946177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:52.959210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3967f-cb4e-49ee-80bf-21454548a197 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:52.993532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:52.998910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:52.999461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:52.999851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.000045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.000774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:53.001083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.003095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0509e-2a73-460a-af7a-0e70e1373fdc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.009590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 178/356] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.010989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.011271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.011542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.031620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:53.033244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:53.076479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1833d-6777-474b-a9c4-d49f1494186d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.087648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.114848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 179/357] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:53.116089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.131590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.132236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.132601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.132922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.135389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.135842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.138767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 179/358] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:52 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.139612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.247907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a774ce0-7690-4c2f-b800-874fb0c7ff95 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.279615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.289406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419374-5ac8-4f28-bbf7-03943f45c83a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.293901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.294778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.295264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.295748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.296681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:53.297280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.298862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.308484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 180/359] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.309555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.336280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.337194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.337471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.337789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.379900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.418965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f197ba-4c12-4049-864f-2ec915a5c063 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.429415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.430765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.457391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.458340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.458834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.460204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.461175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=49615db920c64505958165b68049a95e, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:53.461705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.463618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b874cc-592c-4ca2-a53d-ba13d4e90ffd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.470699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 181/360] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=49615db920c64505958165b68049a95e => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.472302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195a7c1-86f7-4e89-94d6-b05f76406ad7 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.472677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195a7c1-86f7-4e89-94d6-b05f76406ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.473020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195a7c1-86f7-4e89-94d6-b05f76406ad7 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.474311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 182/361] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:53.475085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195a7c1-86f7-4e89-94d6-b05f76406ad7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.536378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.554011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 180/362] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:53.555032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.597360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195a7c1-86f7-4e89-94d6-b05f76406ad7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.637411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.638459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.638989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.639426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] PATH_INFO: `/v3/users/937f24f2314e4f19b6bbfe53d206e76c` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.640921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] RBAC: Authorizing `identity:get_user(user_id=937f24f2314e4f19b6bbfe53d206e76c)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.642700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e1806-c6f8-4a70-b322-72c5a8321440 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.644760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.646056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 183/363] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/users/937f24f2314e4f19b6bbfe53d206e76c => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.647591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.648076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.648546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.671885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:53.673411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:53.698763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa63ef-61f3-4b97-ae11-d46928af9a67 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.739357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.740013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.740110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.740504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.748501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 184/364] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:53.749677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.753916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:53.758996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:53.759869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:53.760905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 181/365] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:53.762491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.804685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.864592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd72a3-84fd-4640-b15e-abe6fb4b577b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.875161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d71be2-f74d-45dd-b327-2bea169957f2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:53.903907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:53.909988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.910896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.911429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.911884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.913409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.915025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.917203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 185/366] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.917518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:53.918224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:53.918460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:53.918690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:53.919165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:53.919987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] RBAC: Adding query filter params (role.id=543bbe978c784ef38f0c36d95b5a194b, user.id=937f24f2314e4f19b6bbfe53d206e76c, scope.system=all) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:53.924963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:53.926532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:53.939923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 182/367] 10.222.0.33 () {56 vars in 1308 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/role_assignments?user.id=937f24f2314e4f19b6bbfe53d206e76c&scope.system=all&role.id=543bbe978c784ef38f0c36d95b5a194b => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:53.940907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.033296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3e3b4-bd1e-4af3-91f3-3473e76333f1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.052125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.076542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77b5e0-85f7-4164-bfbd-9f679ff5914c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.080972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.081829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.082345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.083592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.084483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:54.085222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.086743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.110028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 186/368] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:54.114862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.151508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.152856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.153277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.153687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.154966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.157321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.160042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 183/369] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:53 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:54.168136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.259587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.296746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ebbf16-022c-4281-9e0d-73679ea27360 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.357023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.359086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.359743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.360917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.366956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7efcc2-c71d-4d4f-9fe6-c698ca981187 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.411978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.412962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.413434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.413904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.414825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=2ca946a65b244ca2ab4c18a566425fce) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:54.415456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.417164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.423525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.425532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 184/370] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:54.427509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.457494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.458969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.546057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d15e6-d098-4586-af27-332621b2e664 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.564081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.613708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.615171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.618354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.618928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.619883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 187/371] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:15:54.626928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.642231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:54.652683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:54.653625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:54.654408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:54.657830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 185/372] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:54.665419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.743365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.776227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895a3af3-e1cd-4b21-85d3-bd60817d3e9b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.798504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075951c9-bfcb-4a2f-82b5-7f13fa227eec None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:54.835491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.836536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.837008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.837535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.839028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.842096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.845501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 188/373] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:54.851609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.875436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:54.876289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:54.876805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:54.877329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:54.878304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:54.878811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:54.880405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:54.893837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 186/374] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:54.909619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:54.990826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:54.992288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0c60-1fd7-45ef-9ec1-dca0ed048fa6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.029097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.029739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.030201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.030472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.031604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.033336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.035208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 189/375] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:55.040864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:55.062648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:55.125819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0cb2a-28b6-406e-8316-19a7d9eda20b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.145006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b0a74-560d-4f3f-abf3-5509e2a9d703 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.181773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.182381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.182661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.182982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.183738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=49615db920c64505958165b68049a95e, scope.project.id=2ca946a65b244ca2ab4c18a566425fce) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:55.184034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.185516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.195726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 190/376] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:15:55 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=49615db920c64505958165b68049a95e => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:55.206464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.207507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.207977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.208484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.314161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.316567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.366688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:55.438993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 187/377] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:15:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:15:55.444917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:55.532116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:55.562129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c81ba-b0fb-4ebd-9dfb-366b5d0b0d9b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.598312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.598940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.599256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.599527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] PATH_INFO: `/v3/projects/8012745f80ac4412b0055fc9da17a750` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.600723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] RBAC: Authorizing `identity:get_project(project_id=8012745f80ac4412b0055fc9da17a750)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.602432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.604411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 191/378] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:55 2022] GET /identity/v3/projects/8012745f80ac4412b0055fc9da17a750 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:55.609783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:55.730868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75bf57-7d0d-4c37-ae33-0c6044c60966 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.774454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.775078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.775358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.775623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] PATH_INFO: `/v3/groups/0c30c431e09345009b0516922f282cfb` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.781058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:55.784532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] RBAC: Authorizing `identity:get_group(group_id=0c30c431e09345009b0516922f282cfb)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.785992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0f20-0349-4ca5-9ef8-110f3f65c460 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.788100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 188/379] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:55 2022] GET /identity/v3/groups/0c30c431e09345009b0516922f282cfb => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:55.793723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:55.930357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cfe8f5-95d4-4acd-a7dc-1b08ff305aac None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:55.969763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:55.970382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:55.970655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:55.971472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:55.972201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] RBAC: Adding query filter params (group.id=0c30c431e09345009b0516922f282cfb, role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=8012745f80ac4412b0055fc9da17a750) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:55.972536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:55.974024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e74c4-b74a-441d-b5ac-f36ccb9a127d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:55.981145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 192/380] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:15:55 2022] GET /identity/v3/role_assignments?group.id=0c30c431e09345009b0516922f282cfb&scope.project.id=8012745f80ac4412b0055fc9da17a750&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:56.080684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31924-fca1-4efb-8501-4e4b91056ce3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.081195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31924-fca1-4efb-8501-4e4b91056ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.081195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31924-fca1-4efb-8501-4e4b91056ce3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.082793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 189/381] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:56.088216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed34dd-80c9-4e40-b5e3-d709346424e4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.088544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed34dd-80c9-4e40-b5e3-d709346424e4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.088817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed34dd-80c9-4e40-b5e3-d709346424e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.109313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-86ed34dd-80c9-4e40-b5e3-d709346424e4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:56.110625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-86ed34dd-80c9-4e40-b5e3-d709346424e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:56.114889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:56.192800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 193/382] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:56.241050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff094cc9-94c3-4261-83c0-7b9124ea3807 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.241231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff094cc9-94c3-4261-83c0-7b9124ea3807 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.241506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff094cc9-94c3-4261-83c0-7b9124ea3807 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.243049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 190/383] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:56.248661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.248948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.249202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.273926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:56.275116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:56.358401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 194/384] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:56.551956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff094cc9-94c3-4261-83c0-7b9124ea3807 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:56.658139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff094cc9-94c3-4261-83c0-7b9124ea3807 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:56.696727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:56.697359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.697639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.697910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.710825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:56.715862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:56.716799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:56.717606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 191/385] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:56.723306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:56.808944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:56.836791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:56.887665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0637f0b-564f-4630-8b96-a0c90fc9c1ee None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:56.916512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:56.951743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:56.952395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:56.952669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:56.952933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:56.953691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:56.954015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:56.955509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:56.963469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e5e86-35f3-4a5c-abc3-80425d8bfa7f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:56.974998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 195/386] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:56.981073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.000261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.000915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.001182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.001453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.014081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:57.019274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:57.020164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:57.021006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 192/387] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:57.029774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.164696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693eb942-5580-46c8-8788-366d13ea02b3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.212690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e7a40-29cd-4f72-b1ef-e8d2c3405278 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.221427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:57.238815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.239778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:57.239986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.240583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.241063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.242517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.245274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.248784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 196/388] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:56 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:57.263276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.264166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.264666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.265209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.266044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:57.266518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.268081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.271886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.292348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 193/389] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:57.295130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.421543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba8f4d-a228-4595-8c43-7d38069a829d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.468269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8341e-7e27-43c2-b68d-63f0baa35172 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.469372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.470205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.470620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.471060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.477242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.478877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.481025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 194/390] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:57 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:57.486528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.522697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.523249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.523535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.523812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.524988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.526405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72086bb-f25f-4fd7-93b0-14515d62c3be None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.528361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 197/391] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:57 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:57.529499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e0f7-0f49-4d54-bfc0-f858d2926644 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.529777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e0f7-0f49-4d54-bfc0-f858d2926644 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.530049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0e0f7-0f49-4d54-bfc0-f858d2926644 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.532056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 198/392] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:57.537357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.537637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.537908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.552817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:57.562167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:57.563841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:57.630883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20d81b-0b22-465a-8f6e-98c745c849c7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.652219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:57.660346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 199/393] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:57.660930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.681553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.682179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.682457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.682727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.685921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.685921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.687355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 195/394] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:57 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:57.692850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:57.775415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d5448c-261d-4e92-b537-8a5dd48f91d7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.817736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.818404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.818682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.818949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/groups/a50da47c609a4363862f3a4b08baab16/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.820588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5081c972-a6bd-4583-8922-e846da88d432 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:57.820895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, group_id=a50da47c609a4363862f3a4b08baab16, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.822333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372baa78-6823-49f8-8c17-b1957cdfd859 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.843923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 200/395] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 22 12:15:57 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/groups/a50da47c609a4363862f3a4b08baab16/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:57.849028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:57.886871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:57.887600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:57.887884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:57.888154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/groups/a50da47c609a4363862f3a4b08baab16/roles/49615db920c64505958165b68049a95e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:57.890577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb7ffe4fb56e4683b8f10c494912a706, group_id=a50da47c609a4363862f3a4b08baab16, role_id=49615db920c64505958165b68049a95e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:57.892460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:57.912323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 196/396] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 22 12:15:57 2022] PUT /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706/groups/a50da47c609a4363862f3a4b08baab16/roles/49615db920c64505958165b68049a95e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:15:58.165926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d50ba-94fd-4752-8e19-588feb9a1420 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.166416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.166825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d50ba-94fd-4752-8e19-588feb9a1420 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.167257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d50ba-94fd-4752-8e19-588feb9a1420 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.168753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 201/397] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:58.175705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.175989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.176547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.200965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:58.202158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:58.267248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:58.286431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 202/398] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:58.302238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c47dc3-ec58-415d-b429-64813981f3b9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:58.341937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:58.342567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.342838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.343101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.357494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:58.362953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:58.363829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:58.365069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 197/399] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:15:58.370337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.502902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142d769-c972-4ac2-9730-7662ed4bc632 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:58.550107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:58.550726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.551000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.551264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.551918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:58.552211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:58.553721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:58.568756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 203/400] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:58.574229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.595103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.626096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:58.672659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:58.708703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55e144-9d4d-4b6e-8a8b-2331a2150657 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:58.749063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c4e01-326f-4fa0-900d-411e20f1a5b5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:58.755778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:58.756381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.756659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.756927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.760950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:58.762392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:58.763961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 198/401] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:58.770001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.795684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:58.796333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:58.796614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:58.796882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:58.818027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:58.823363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:58.824224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:15:58.825050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:15:58.826832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 204/402] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:58.833734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:58.987286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fc7e1-6e9e-42f6-87f3-e9a2f2bcc700 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:58.989481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f687b4c-e532-4bb7-b35d-c4b063546a7e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.008967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:59.010240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:59.027119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.027812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.028228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.028488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.029288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:15:59.029698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.031141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.042869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 205/403] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:59.049286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:59.089006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.089648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.089929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.090205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.091355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.092824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.094750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 199/404] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:58 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:59.099921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:59.166833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440e9f-70e5-4378-9a32-c224da53a902 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.210716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.211347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.211616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.211881 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.213276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.214756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9cdcc-99a6-421f-b7bc-fbd373978057 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.216669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 206/405] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:15:59 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:15:59.217859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770070-907b-406e-b809-2cbfabb81b51 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.218129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770070-907b-406e-b809-2cbfabb81b51 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.218390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70770070-907b-406e-b809-2cbfabb81b51 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.223810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 207/406] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:15:59.224767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770070-907b-406e-b809-2cbfabb81b51 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:59.248588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b6602-1d51-4983-a57b-5d55a39a3e03 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.271002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:59.298488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.299666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.299945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.300483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/groups/a50da47c609a4363862f3a4b08baab16/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.303284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, group_id=a50da47c609a4363862f3a4b08baab16, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.304753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33936377-9956-4b9a-bf13-03ddafe160b8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.325013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 200/407] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 22 12:15:59 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/groups/a50da47c609a4363862f3a4b08baab16/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:15:59.326395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.326682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.326954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.350023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:15:59.351595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:15:59.352972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70770070-907b-406e-b809-2cbfabb81b51 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.387349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:59.425413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.426055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.426382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.426652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.431304 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.432844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.434622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 208/408] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:15:59 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:15:59.437640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 201/409] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:15:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:15:59.441641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:59.554369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a270c8f-87af-49d7-90da-af7bab55c3f9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.564919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:15:59.591351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.592019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.592312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.592586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce/groups/a50da47c609a4363862f3a4b08baab16/roles/49615db920c64505958165b68049a95e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.594721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] RBAC: Authorizing `identity:create_grant(project_id=2ca946a65b244ca2ab4c18a566425fce, group_id=a50da47c609a4363862f3a4b08baab16, role_id=49615db920c64505958165b68049a95e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:15:59.596787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:15:59.614038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 209/410] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 22 12:15:59 2022] PUT /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce/groups/a50da47c609a4363862f3a4b08baab16/roles/49615db920c64505958165b68049a95e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:15:59.781822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:15:59.904158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079dfa43-1232-4906-9953-e966d3937dc4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:15:59.938638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:15:59.939296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:15:59.939619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:15:59.939892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:15:59.949871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:15:59.954604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:15:59.956071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:15:59.957101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 202/411] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:15:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:15:59.961937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:00.079774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fbbd5-4761-4a3e-b01c-c0e27da15484 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:00.123737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:00.124341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.124625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.124947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.125610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:00.125907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:00.127341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:00.138984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 210/412] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:15:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:00.145496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:00.180960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:00.276244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964797b-9066-4bae-bf32-452bc0f4a23e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:00.311956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:00.313453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.313941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.314934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] PATH_INFO: `/v3/groups/0c30c431e09345009b0516922f282cfb` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.316235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] RBAC: Authorizing `identity:get_group(group_id=0c30c431e09345009b0516922f282cfb)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:00.317910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:00.320025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 203/413] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:00 2022] GET /identity/v3/groups/0c30c431e09345009b0516922f282cfb => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:00.325263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:00.370527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:00.465602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42777d61-6da0-4682-bcdb-82a0a2ebf5b9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:00.499264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:00.499904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.500181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.500481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] PATH_INFO: `/v3/projects/8012745f80ac4412b0055fc9da17a750` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.501647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] RBAC: Authorizing `identity:get_project(project_id=8012745f80ac4412b0055fc9da17a750)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:00.503615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fd063-c4de-4717-aad9-307c2b82a04d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:00.505625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 211/414] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:00 2022] GET /identity/v3/projects/8012745f80ac4412b0055fc9da17a750 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:00.511015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:00.654582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbe6f-b5c9-4cca-a1bc-7373633af499 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.654912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbe6f-b5c9-4cca-a1bc-7373633af499 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.655122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03fbe6f-b5c9-4cca-a1bc-7373633af499 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.656347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:00.656714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 212/415] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:00.661985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ae1fd-c131-4e9e-8340-f429bcc8befb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.662269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ae1fd-c131-4e9e-8340-f429bcc8befb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.662544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ae1fd-c131-4e9e-8340-f429bcc8befb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.666420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa1485-dc5c-4dab-b20a-9264f2c693ef None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:00.700385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:00.701071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.701347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.701618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] PATH_INFO: `/v3/projects/8012745f80ac4412b0055fc9da17a750/groups/0c30c431e09345009b0516922f282cfb/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.703585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] RBAC: Authorizing `identity:create_grant(project_id=8012745f80ac4412b0055fc9da17a750, group_id=0c30c431e09345009b0516922f282cfb, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:00.705076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbc0d6-48a9-485e-9a5b-93e69db7d028 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:00.705322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d25ae1fd-c131-4e9e-8340-f429bcc8befb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:00.706505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d25ae1fd-c131-4e9e-8340-f429bcc8befb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:00.729776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 204/416] 10.222.0.33 () {58 vars in 1442 bytes} [Thu Sep 22 12:16:00 2022] PUT /identity/v3/projects/8012745f80ac4412b0055fc9da17a750/groups/0c30c431e09345009b0516922f282cfb/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:00.803151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 213/417] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:00.953346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4ae9-8527-46a9-b602-b02f8127800b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.953696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4ae9-8527-46a9-b602-b02f8127800b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.953893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d4ae9-8527-46a9-b602-b02f8127800b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:00.955522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 205/418] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:00.974559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:00.974874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:00.975171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.001356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:01.003067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:01.033649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:01.085496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 214/419] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:01.096374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d4ae9-8527-46a9-b602-b02f8127800b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:01.212809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d4ae9-8527-46a9-b602-b02f8127800b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:01.253329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:01.253878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:01.254149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:01.254469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.264529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:01.269717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:01.270595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:01.271401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 206/420] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:01.276590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:01.366634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:01.400680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:01.431679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d0d94-48c8-4137-b8f4-4d6f04cc7587 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:01.510480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:01.511469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:01.511949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:01.512452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.517959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:01.519768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:01.524750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:01.543120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 215/421] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:01.549101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:01.562195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d744f2-8bbe-4dcd-ac4f-402988358b4e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:01.593002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:01.601128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:01.602357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:01.602854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:01.603577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.614750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:01.620204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:01.621142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:01.621930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:01.628670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 207/422] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:01.630561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:01.739122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd10072-aedd-4477-beb9-bf7782aeae27 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:01.784422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:01.784422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:01.784422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:01.784422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.784787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:01.840150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b35ef1-e6f7-4821-9151-18d950b5e11a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:01.901233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:01.905703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1e3a9-b913-41a2-8bce-7edde58f7bb4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:01.921046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:01.921672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:01.921952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:01.922223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:01.922896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:01.923202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:01.924631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:01.938700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:01.943121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 208/423] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:01.949253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.052354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 216/424] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:02.058034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6a20-3d27-4cb4-80e2-bf0d8ee1ef76 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.058393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6a20-3d27-4cb4-80e2-bf0d8ee1ef76 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.058808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576f6a20-3d27-4cb4-80e2-bf0d8ee1ef76 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.060068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 217/425] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:02.060643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6a20-3d27-4cb4-80e2-bf0d8ee1ef76 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.075264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b575c66-341f-4519-aaaa-b0c9c2fa5872 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.112851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.113828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.114296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.114787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.117579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.169295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576f6a20-3d27-4cb4-80e2-bf0d8ee1ef76 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.191191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.191358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.200314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.216432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.217171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.217449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.217715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.218909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.220553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c8c09-5a8c-4d6d-b85e-393a69a4ba92 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.232138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 218/426] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:02.232138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.232138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.232138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.252610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:02.254102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:02.292033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 209/427] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:02.292978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.329729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.336193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 219/428] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:02.338288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.398181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3023a8f-4f11-4043-a52b-c97a9e59312e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.432505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.433122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.433391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.433657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.434768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.436166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fe0d8-e0e2-4c3e-af21-0701f96e2afc None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.437730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 210/429] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:02.438824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6962f-822c-431e-84fc-8853338efc6b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.439657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6962f-822c-431e-84fc-8853338efc6b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.439934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6962f-822c-431e-84fc-8853338efc6b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.441352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 211/430] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:02.443885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6962f-822c-431e-84fc-8853338efc6b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.462846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af0ff7-d69b-4376-9039-e0811a67f08b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.485753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.512328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.512328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.512328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.512328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] PATH_INFO: `/v3/projects/eb7ffe4fb56e4683b8f10c494912a706` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.512727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] RBAC: Authorizing `identity:get_project(project_id=eb7ffe4fb56e4683b8f10c494912a706)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.514888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09566e7a-6447-4c87-ab77-64c52bdbf38c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.515921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 220/431] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/projects/eb7ffe4fb56e4683b8f10c494912a706 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:02.517593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.518000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.518599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.542345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:02.543727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:02.558590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6962f-822c-431e-84fc-8853338efc6b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.560217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.595017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.596145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.596449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.596716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.597479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:02.597780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.599225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.605734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 212/432] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:02.606584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.638036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 221/433] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:02.679699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.721682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608872bf-fd3e-4a06-b20d-17e6a804b406 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.759724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.760441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.760716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.760983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.762081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.763525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.765556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 213/434] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:02.776811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.790906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe2b1b-a205-48ba-8cc7-81a5c2d18bae None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.809872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.829459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.830499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.830974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.831469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.843874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:02.844264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:02.850349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:02.851279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 12:16:02.852466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:02.855097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 222/435] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:02.862310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:02.932660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24749f01-511a-4e98-a907-98089ae59274 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.969219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:02.969822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:02.970096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:02.970933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:02.971645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=49615db920c64505958165b68049a95e, scope.project.id=eb7ffe4fb56e4683b8f10c494912a706) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:02.971947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:02.972759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a90e-bb6d-4f04-8351-90548b9f8268 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:02.973403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:02.979877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 214/436] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=eb7ffe4fb56e4683b8f10c494912a706&role.id=49615db920c64505958165b68049a95e => generated 611 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:03.010490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.011492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.011986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.012479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.013358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] RBAC: Adding query filter params (name=member) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:03.013874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.015490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.031100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 223/437] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:16:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:03.032139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.058466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.107170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.180396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.185255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb6c78-b49d-4427-9b5f-9b702a612ace None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.212212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8854ec9-4a18-4564-9f6b-d789919be539 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.236812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.237593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.237950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.238312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.254004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.254686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.255101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.255380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.268562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:03.273872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:03.274755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:03.275569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 224/438] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:03.280891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.322877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.324358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.402329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.406657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061fae7-d986-455c-b46f-708446d1fb07 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.446195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.446818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.447096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.447368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.448679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:03.448679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.449943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.457871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.466942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 225/439] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:03.467138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 215/440] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:03.468135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.473993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.591338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd506c-d4c2-40a1-9ead-da8a1b5ecab1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.601165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550f43-4d03-4a03-b805-b4fd5961d6c8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.630408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.631016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.631293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.631562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.636083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.636749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.637024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.637296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.638407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.639840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6203daea-6271-496b-929d-a19955495ca0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.641697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 216/441] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:03.643158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d9a-75ec-4cfc-b403-1a19b59888a5 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.643436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d9a-75ec-4cfc-b403-1a19b59888a5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.643703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f897d9a-75ec-4cfc-b403-1a19b59888a5 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.645055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 217/442] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:03.650321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.650573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.650838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.662996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.673100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:03.674331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:03.702664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.704126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.732401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.739869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 218/443] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:03.740696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.806543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 226/444] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:16:03.807608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.848667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9ce89-b38f-4d87-a3ca-5e2845f17474 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.866866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:03.901579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.902201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.902472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.902735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.903824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.905298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.907004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 219/445] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:03.912975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.913462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c49f71-df98-430b-9d84-f3112f569c60 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:03.947132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:03.947892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:03.948238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:03.948545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] PATH_INFO: `/v3/projects/2ca946a65b244ca2ab4c18a566425fce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:03.949659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] RBAC: Authorizing `identity:get_project(project_id=2ca946a65b244ca2ab4c18a566425fce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:03.951114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:03.952906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 227/446] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/projects/2ca946a65b244ca2ab4c18a566425fce => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:03.958856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:03.975617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.017848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f32665-5ef8-41cc-a0b6-d8f697a7dc0b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.050485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.051081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.051360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.051634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.052371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=0568cdc72a2e49fb931bbf5cf5be87e5, scope.project.id=2ca946a65b244ca2ab4c18a566425fce) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:04.052682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.054107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.059982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 220/447] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=0568cdc72a2e49fb931bbf5cf5be87e5 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:04.072642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.072732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6b260-8d48-46b1-9860-41d32bea66de None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.096065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.105839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.106477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.106751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.107012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] PATH_INFO: `/v3/groups/a50da47c609a4363862f3a4b08baab16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.108140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] RBAC: Authorizing `identity:get_group(group_id=a50da47c609a4363862f3a4b08baab16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.109633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.111314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 228/448] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:03 2022] GET /identity/v3/groups/a50da47c609a4363862f3a4b08baab16 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:04.117440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.191132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5549850-16d1-41ed-9e7a-9b2223d794a0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.227289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.236939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.237928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.237928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.238298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.241199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80cd37-6744-41db-9e72-74beac3e332f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.252117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:04.257139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:04.258030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:04.259165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 221/449] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:04.264044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.285518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.286231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.286950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.286950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.287487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] RBAC: Adding query filter params (group.id=a50da47c609a4363862f3a4b08baab16, role.id=49615db920c64505958165b68049a95e, scope.project.id=2ca946a65b244ca2ab4c18a566425fce) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:04.287789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.289244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.295272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 229/450] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/role_assignments?group.id=a50da47c609a4363862f3a4b08baab16&scope.project.id=2ca946a65b244ca2ab4c18a566425fce&role.id=49615db920c64505958165b68049a95e => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:04.389517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04cd6b-d927-40b8-9105-76786ec2c7b4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.429430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.430045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.430320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.430593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.431265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:04.431579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.432977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.444051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 222/451] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:04.450005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.511626 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.553576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e50a4-186a-45fb-a137-ef67e0892724 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.586462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.587056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.587331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.587599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.653586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.655028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.709769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.749119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 230/452] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:04.755379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.803359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:04.851031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a0bad5-a6f7-42a3-ac18-ac3a2f7b10ff None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:04.886459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:04.887074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:04.887346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:04.887612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] PATH_INFO: `/v3/projects/8012745f80ac4412b0055fc9da17a750` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:04.888718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] RBAC: Authorizing `identity:get_project(project_id=8012745f80ac4412b0055fc9da17a750)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:04.890116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:04.891704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 223/453] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/projects/8012745f80ac4412b0055fc9da17a750 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:04.897667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:04.993131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfb5da-6209-450d-be5e-6a2a022558fd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:05.024823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:05.025433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:05.025710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:05.025982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] PATH_INFO: `/v3/groups/0c30c431e09345009b0516922f282cfb` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:05.027071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] RBAC: Authorizing `identity:get_group(group_id=0c30c431e09345009b0516922f282cfb)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:05.028527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca2dc63-54c9-4a26-a15f-3f3d6d2f1248 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:05.030066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 231/454] 10.222.0.33 () {56 vars in 1180 bytes} [Thu Sep 22 12:16:04 2022] GET /identity/v3/groups/0c30c431e09345009b0516922f282cfb => generated 229 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:05.036421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:05.129817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d1e7-68f3-4e04-9343-2c50936cdc15 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:05.145934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:05.162532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:05.163114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:05.163385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:05.163651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:05.164374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] RBAC: Adding query filter params (group.id=0c30c431e09345009b0516922f282cfb, role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=8012745f80ac4412b0055fc9da17a750) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:05.164675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:05.166079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3bbda-638b-4312-98e1-6a7dba07b18d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:05.171494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 224/455] 10.222.0.33 () {56 vars in 1376 bytes} [Thu Sep 22 12:16:05 2022] GET /identity/v3/role_assignments?group.id=0c30c431e09345009b0516922f282cfb&scope.project.id=8012745f80ac4412b0055fc9da17a750&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 611 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:07.665423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0450e-048a-48e8-bd95-ab33db57d432 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:07.665970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0450e-048a-48e8-bd95-ab33db57d432 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:07.665970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0450e-048a-48e8-bd95-ab33db57d432 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:07.667549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 232/456] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:07.682149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b22e-222e-4808-8265-6462c73d8752 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:07.683029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b22e-222e-4808-8265-6462c73d8752 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:07.683520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289b22e-222e-4808-8265-6462c73d8752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:07.704078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9289b22e-222e-4808-8265-6462c73d8752 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:07.706079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9289b22e-222e-4808-8265-6462c73d8752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:07.796642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 225/457] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:07.954075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d07fb-cad4-4e09-955d-918295af942a None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:07.955216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d07fb-cad4-4e09-955d-918295af942a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:07.955701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d07fb-cad4-4e09-955d-918295af942a None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:07.957498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 233/458] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:07.962917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e4d72-e445-4127-832f-0adbd77bd805 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:07.963434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e4d72-e445-4127-832f-0adbd77bd805 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:07.963924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e4d72-e445-4127-832f-0adbd77bd805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:07.987977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-051e4d72-e445-4127-832f-0adbd77bd805 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:07.989599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-051e4d72-e445-4127-832f-0adbd77bd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:08.084426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 226/459] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:08.135152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4de7a-56ab-43ba-9048-39750549e24c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.135152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4de7a-56ab-43ba-9048-39750549e24c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.135152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd4de7a-56ab-43ba-9048-39750549e24c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.136133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 234/460] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:08.141767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.142603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.142876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.168657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:08.169833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:08.215547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:08.256816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 227/461] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:08.295468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4de7a-56ab-43ba-9048-39750549e24c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.324419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:08.405580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd4de7a-56ab-43ba-9048-39750549e24c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:08.453995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:08.454648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.454922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.455186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.475878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:08.477543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:08.498348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:08.499327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:08.500193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:08.500193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.500193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:08.500193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 235/462] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:08.506413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.570285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.639356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfa3bf-8d0e-42e9-bcc9-dd82c112555d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:08.665780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:08.676348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:08.676946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.677219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.677485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.678277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:08.678823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:08.679920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:08.689286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27286fa1-7f97-43b5-8a95-ec2f9a19370f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:08.695643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 228/463] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:08.695643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.726411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:08.727211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.727479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.727739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.731961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:08.739330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:08.740814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-8db88855-e5b2-4e7a-9208-29eaa061ddd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:08.754825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:08.755738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:08.756637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:08.756637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.756637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:08.756637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 236/464] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:08.761760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670934b-bbc3-4df0-9510-3e32d8b1ddc9 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.762038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670934b-bbc3-4df0-9510-3e32d8b1ddc9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.762309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670934b-bbc3-4df0-9510-3e32d8b1ddc9 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.763522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 237/465] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:08.764461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670934b-bbc3-4df0-9510-3e32d8b1ddc9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.808969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e7796-ccba-409f-9631-7244a94ff8ff None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:08.842893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:08.843518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.843795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.844067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.855449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:08.856928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:08.876870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670934b-bbc3-4df0-9510-3e32d8b1ddc9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:08.878446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:08.879348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:08.880172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:08.880172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:08.880172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:08.880172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 229/466] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:08.880431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:08.912210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:08.917734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:08.918337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.918614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.918885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.919523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:08.920979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b5c22-13db-4040-ada6-560245a1ab07 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:08.944764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 238/467] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:08 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:08.945993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:08.946276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:08.946549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:08.966801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:08.968150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:08.997101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:09.017946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc62cf-38ee-4096-b90a-39e3acbdd0c4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.051554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 239/468] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:09.057469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.061057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.061973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.062478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.062944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.063842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:09.064362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.067138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.082908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 230/469] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:09.092374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.205210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e90630c-82f3-4421-84cd-1fe56fb1b1c6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.249011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.249457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ef29d-c1f4-4858-b7a1-7086c70d9d8d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.250020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.250435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.250843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.252148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.253809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.273295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:09.274950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:09.284641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 231/470] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:09 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:09.311496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.312606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.313126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.314221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.315255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:09.315865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.317731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.342452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 240/471] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:09.349609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.411655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.478048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3424af-0321-4a8e-86d3-5eb0f53c476a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.533815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.534335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.534613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.534880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.535530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.536983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a54839-cc04-4af5-b932-9e7e14381659 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.542783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d99db-0fc9-4774-a229-d4658b4ba830 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.559509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 232/472] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:09 2022] POST /identity/v3/users => generated 248 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:09.560588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad909-c268-4762-8fc5-fa406ed37c6a None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.560882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad909-c268-4762-8fc5-fa406ed37c6a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.561145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bad909-c268-4762-8fc5-fa406ed37c6a None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.562525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 233/473] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:09.568529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.568810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.569081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.592535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:09.593812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:09.618817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.619771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.620182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.620625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.633058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.634861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:09.653857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:09.654992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:09.656007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:09.656007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:09.656007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:09.656618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:09.664012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 241/474] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:09.670078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.684183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 234/475] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:09.707745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:09.786850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485bb23-d693-4c4b-a0e3-e516a8b25cf9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.821986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:09.822942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:09.823416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:09.823883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:09.824737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:09.825253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:09.826807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:09.840155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 242/476] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:09.846574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:09.970474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c2031-58ce-4856-a502-86674e2bb0d6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:09.987379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:10.012822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09354266-e267-49a2-951a-196aea421d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:10.013421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354266-e267-49a2-951a-196aea421d11 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:10.013704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354266-e267-49a2-951a-196aea421d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:10.013972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354266-e267-49a2-951a-196aea421d11 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:10.014823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09354266-e267-49a2-951a-196aea421d11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:10.017383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09354266-e267-49a2-951a-196aea421d11 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:10.053921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 235/477] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:09 2022] POST /identity/v3/users => generated 249 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:10.117360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:10.129320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c8400-0228-48ed-b3e2-290daf0579a3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:10.170809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:10.171440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:10.171709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:10.171971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:10.172743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:10.174196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:10.190852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.sql.core [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 12:16:10.190852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 12:16:10.190852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [parameters: {'id': 'a7f091d00078416da102b61d1f50512a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 12:16:10.190852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result.read() Sep 22 12:16:10.214008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:16:10.214945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application yield resource Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise self.value Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application session.commit() Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 12:16:10.215856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.session.flush() Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application compat.raise_( Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 12:16:10.217862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:16:10.219979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result.read() Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:16:10.226401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [parameters: {'id': 'a7f091d00078416da102b61d1f50512a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:10.228607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 12:16:10.230520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:10.232527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 243/478] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 22 12:16:09 2022] POST /identity/v3/roles => generated 144 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:10.232661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09354266-e267-49a2-951a-196aea421d11 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:10.313182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:10.382225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09354266-e267-49a2-951a-196aea421d11 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:10.416699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:10.417331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:10.417609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:10.417883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:10.431049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-76aae66d-2576-4117-b27c-2102e20e6c4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:10.436364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:10.437264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:10.439202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 236/479] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:10.443895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:10.552286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c218b2c-1b9e-43b7-b3d1-4fa0d84008cc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:10.588107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:10.588777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:10.589112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:10.589515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:10.590229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:10.590529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:10.591958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf254b5c-8d2f-496a-85bb-9d8b8c416853 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:10.606839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 244/480] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:11.750181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c49-17dc-42bc-a4d6-54a17eca862e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:11.750727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c49-17dc-42bc-a4d6-54a17eca862e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:11.750727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c1c49-17dc-42bc-a4d6-54a17eca862e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:11.752854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 237/481] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:11.758157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6842f9-f2b6-47f3-9111-69851bc6c29b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:11.758548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6842f9-f2b6-47f3-9111-69851bc6c29b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:11.758729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6842f9-f2b6-47f3-9111-69851bc6c29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:11.789332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1d6842f9-f2b6-47f3-9111-69851bc6c29b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:11.790543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1d6842f9-f2b6-47f3-9111-69851bc6c29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:11.802045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:11.880521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 245/482] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:12.227922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd5b6-1717-4344-a040-7206a15de57b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.228619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd5b6-1717-4344-a040-7206a15de57b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.229093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dfd5b6-1717-4344-a040-7206a15de57b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.230925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 238/483] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:12.236359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c203d2-0dea-4e7c-87be-e06a9495b356 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.236854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c203d2-0dea-4e7c-87be-e06a9495b356 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.237335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c203d2-0dea-4e7c-87be-e06a9495b356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.262526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f9c203d2-0dea-4e7c-87be-e06a9495b356 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:12.264077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f9c203d2-0dea-4e7c-87be-e06a9495b356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:12.361217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfd5b6-1717-4344-a040-7206a15de57b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:12.363014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 246/484] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:12.428833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed81721-a1f3-405a-adb3-3aa663b4006f None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.430106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed81721-a1f3-405a-adb3-3aa663b4006f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.430481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed81721-a1f3-405a-adb3-3aa663b4006f None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.432171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 247/485] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:16:12.439037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.439316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.439589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.483977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:12.485099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:12.539340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dfd5b6-1717-4344-a040-7206a15de57b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:12.542343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:12.608873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:12.609495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.609768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.610030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.620102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 248/486] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:12.625958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:12.631241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:12.632114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:12.633343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 239/487] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:12.641373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:12.656236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:12.657051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:12.813098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c47b30-1954-48bc-8dde-6addc0bafafa None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:12.840246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a42c7-58bd-401a-ae25-24d22dac08f2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:12.893261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:12.893866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.894199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.894468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.895120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:12.895414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:12.896812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:12.905928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 249/488] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:12.922426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:12.936950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:12.937582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:12.937853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:12.938121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:12.952622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:12.959850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:12.960801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:12.961617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 240/489] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:12.962290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:12.991992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.011402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.049375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173aa930-7796-4397-9e68-c0467a93e8e8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.084278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.085230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.085503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.085763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.091983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901467e-a306-44fb-b0e2-fac11f56fdd5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.096432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.097924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:13.112814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:13.113903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:13.114950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:13.114950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.114950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:13.118595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 250/490] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:13.119434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.134833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.135635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.135912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.136181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.148983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:13.155165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:13.156230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:13.158058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:13.159978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 241/491] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:13.161330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.236433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f46d2b-a130-4f70-936b-340138f3903d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.256093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.262612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.276477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bd3f4-5a17-4613-9a2c-54ff3899dec8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.278573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.279177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.279454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.279726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.280415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:13.280719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.282164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6cd28f-1492-41d0-8abb-66bb0c8781a2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.290353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 251/492] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:13.291528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ea30f-0216-4276-b08a-1ebb2f743c07 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.291811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ea30f-0216-4276-b08a-1ebb2f743c07 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.292085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ea30f-0216-4276-b08a-1ebb2f743c07 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.297313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 252/493] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:13.298116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ea30f-0216-4276-b08a-1ebb2f743c07 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.318308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.319172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.319616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.320126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.320965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:13.321260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.322628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.335237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 242/494] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:13.335984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.403446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ea30f-0216-4276-b08a-1ebb2f743c07 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.433720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7a20-bd50-4df2-957f-fbe9cff9583a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.438282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.439481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.439754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.440015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.440696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:13.440995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.442372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d1f42-c07f-4ff4-b626-ef0de20add8e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.452165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 253/495] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:13.453420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.453705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.453975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.467672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.468277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.468578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.468842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.471178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:13.472397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:13.479014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.480471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.482171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-1b62a16c-cf74-4827-b056-db5441d93a1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:13.497271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:13.498358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:13.499390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:13.499390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.499390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:13.499390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 243/496] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:13.499653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67cdcd-a1d4-4cde-87d8-9c93c7845908 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.499729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67cdcd-a1d4-4cde-87d8-9c93c7845908 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.499981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67cdcd-a1d4-4cde-87d8-9c93c7845908 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.501965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 244/497] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:13.502518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67cdcd-a1d4-4cde-87d8-9c93c7845908 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.520345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.546620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 254/498] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:13.547433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.626610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67cdcd-a1d4-4cde-87d8-9c93c7845908 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.646609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14953d-c768-4037-a9cd-228d414f48b7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.662004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.662722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.662999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.663361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.670856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.672589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.680214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.680904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.681182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.681452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:13.686659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:13.687737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:13.689538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:13.695969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 245/499] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:13.695969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.696171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.696171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-39a71e37-3d4e-4376-b052-81200e8eebcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:13.710732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:13.711808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:13.713347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:13.713347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:13.713347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:13.716574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 255/500] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:13.717627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.717906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.718176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.736406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:13.737633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:13.755199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.759063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:13.804461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e130f32-33ff-4324-b44e-113767b281a0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.811894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 256/501] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:13.813473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.842127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476ee4-092d-49d8-8553-225493226fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.843081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476ee4-092d-49d8-8553-225493226fac None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.843460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476ee4-092d-49d8-8553-225493226fac None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.843826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88476ee4-092d-49d8-8553-225493226fac None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.845085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476ee4-092d-49d8-8553-225493226fac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:13.845457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476ee4-092d-49d8-8553-225493226fac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.846884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88476ee4-092d-49d8-8553-225493226fac None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.863403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 246/502] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:13.864134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476ee4-092d-49d8-8553-225493226fac None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:13.923516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a895aa-1fa3-4e5e-a8df-e3229d150338 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.957255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:13.957844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:13.958125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:13.958383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:13.959083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:13.959378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:13.960831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:13.973274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88476ee4-092d-49d8-8553-225493226fac None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:13.987986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.002841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 257/503] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.002841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.014589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.015190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.015469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.015744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.016422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:14.016728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.017177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.018273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.034807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 247/504] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.034807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.109866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a436a-acc5-4f79-bebe-2121af54da5d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.136593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9337a4-4350-4a7f-9a6e-06bdf62e9b8c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.142845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.143468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.143742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.144004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.153700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:14.158980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:14.160029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 258/505] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:14.161013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.171827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.172487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.172821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.173096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.182454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.184439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:14.195843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:14.196930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 248/506] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:14.197809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.232718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.237031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.261495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51871bcb-c164-45c6-9454-f9b5be39a65e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.295235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.295827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.296102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.296396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.297420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd705f7-7303-4ef8-b365-fe7f2963c84e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.330055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.330986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.331265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.331530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.338601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.340015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:14.351254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:14.352593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 249/507] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:14.354046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.367087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.368579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.439668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.444866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.452388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e4a45-55e9-4d36-b486-fcce56ce7ad6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.465523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 259/508] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:14.466404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.489734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.490352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.490624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.490890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.501756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.503188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:14.517548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:14.518658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:14.519522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:14.519522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:14.519522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:14.519522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 250/509] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:14.519522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.570233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e58a7-d996-4ff4-bc72-f1fbbe9647c8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.604342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.604936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.605206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.605469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.606108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:14.606403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.607763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.616054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dedb02-9911-4eb5-b2b1-28b09e4ff8dc None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.616571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 260/510] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.617350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.649054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.649643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.649915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.650182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.650883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:14.651180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.652611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.653578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.688847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 251/511] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.689659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.693002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.721984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c24773-bd35-4ad6-9912-9d0bc9ab612e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.754666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.755261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.755535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.755801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.756534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:14.756832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.758251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.775307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 261/512] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.776187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.787213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61c80d-8ec3-4801-b061-b367eb2ee7ec None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.820074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.820782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.821053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.821317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.831308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.832809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:14.846713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:14.848153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:14.849196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:14.849196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:14.849196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:14.849196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 252/513] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:14.849196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.875870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.882883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc3936-8c96-4677-ab1e-e992ffc55afa None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.904918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:14.916597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.917197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.917466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.917729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.918346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:14.918643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.920011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:14.930800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 262/514] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:14.931647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:14.952824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fcde19-7448-4f92-a56b-3aad796e2130 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:14.985651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:14.986258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:14.986530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:14.986795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:14.997227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:14.998690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:15.012381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:15.013306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:15.014152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:15.014152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.014152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:15.014152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 253/515] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:15.014370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.030034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92274e5d-46b1-49aa-83d5-270f519e6e9a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.062133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.062728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.063005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.063300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.086795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.112639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e041735-3069-475b-beb5-2c6dc2671d0f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.121454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.130894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.132355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.146352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.147100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.147380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.147646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.211568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.212960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.227613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 263/516] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:15.228457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.282833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.307673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 254/517] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:15.308511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.318542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.328544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b052127-c12a-4195-bc34-49b64bbf0725 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.361594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.362191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.362467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.362736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.363361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:15.363658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.365067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.376356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 264/518] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:15.377275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.406734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80053b-5393-4228-9667-95bb6eb3ad97 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.439494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.440074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.440377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.440695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.441319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.442738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.462958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 255/519] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:15 2022] POST /identity/v3/users => generated 247 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:15.463681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.474818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cc996-ad46-409c-8209-5e8742901274 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.492457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.525834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.526995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.527983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.528444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.529498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:15.529945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.532184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.544329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.550083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 265/520] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:15.551288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.591921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bddaf-5b95-4de0-bcf3-b132f0856324 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.631009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.631635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.631911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.632175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.645368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.646983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:15.661289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:15.662197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:15.663041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:15.663041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.663041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:15.663041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 256/521] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:15.663252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.671002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fc798-d6b9-438d-976d-84c29454cf7b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.709904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.710534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.710806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.711072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.721423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.722781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:15.737654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:15.738571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:15.739429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:15.739429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:15.739429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:15.739429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 266/522] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:15.739659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.770755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.778690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac413e-584e-4ab1-8526-58080635d7b6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.795157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:15.815339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.816051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.816418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.816758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.823548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.825121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:15.837314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:15.838375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:15.839438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:15.840320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 257/523] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:15.840913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.852969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73de95-e6eb-4f09-a0c2-8d0aadb6e5f8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.891612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.892238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.892544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.892813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.902369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.903817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:15.916676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:15.917698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:15.918584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 267/524] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:15.921144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:15.957174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b508ee2e-3d48-43e9-a1c6-48a01292f096 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:15.993705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:15.994225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:15.994503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:15.994774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:15.995408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:15.995706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:15.997114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.009552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 258/525] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.010364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.024564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.039807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4574b81-d71b-446f-a8a6-5778a0bf54e0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.051351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.074735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.075491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.075815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.076207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.076907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.077208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.078692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.095573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 268/526] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.095573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.121794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fb6ae6-b3a2-4267-ad17-416dcbf0918d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.157914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.158514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.158787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.159058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.159739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.160038 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.161501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.180692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 259/527] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.181549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.204571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771e4a72-aa49-43ce-a0fb-98a619eb7bdd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.244368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.245326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.245892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.246175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.246877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.247178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.248635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.268602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 269/528] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.273232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.276210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.280980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.290399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0709-394a-43c2-a4d3-bba629a4d378 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.325600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.326306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.326649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.326980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.333499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.334916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:16.346717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:16.347752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 260/529] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:16.348660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.386589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54b919-03ed-41dc-a687-a83087ce0c27 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.422033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.423230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.423513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.423777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.430469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.432016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:16.444856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:16.445867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 270/530] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:16.446743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.460764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0c243-dd10-4266-b6f5-1e56683c620b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.496858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.497365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.497685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.498063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.499076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.499387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.501381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.507176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 261/531] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.507176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.521500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.540067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.574419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06903d4-221c-4561-8d64-2de295c88674 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.619515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.620598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.620876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.621440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.646275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4f95e-bc29-4ebb-a7f6-a07268d9d80b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.695822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.696861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.697360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.697682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.698373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.698729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.700205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.714301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 262/532] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.715150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.720683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.722191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.801182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.806143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:16.828849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fa0f7-ef29-48e9-950e-8385d91b2fe2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:16.859890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 271/533] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:16.859890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.889302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:16.890242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:16.890746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:16.891236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:16.892251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:16.892779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:16.894423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:16.911892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 263/534] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:16.913094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:16.989185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be009f0-b28b-4c51-bc83-c45042f88047 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.026557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880bc7c1-545f-46d6-891c-a5b062f984b6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.032748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.033631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.034107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.034576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.035487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=628750c611c24e8b9b4c0e66d2e3282e) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:17.035996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.037772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.043839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 272/535] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/role_assignments?user.id=628750c611c24e8b9b4c0e66d2e3282e&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:17.044907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:17.069137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.069690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.069967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.070240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.079052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:17.083386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:17.085985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.089956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:17.110004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:17.111484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:17.112374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:17.112374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:17.112374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:17.112374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 264/536] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:17.122324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:17.171533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93606300-8552-45b1-8287-0171c2a00ccc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.208671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.209666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.210205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.210673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.211629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=e10e2b0b66754f128cddc94ca3daf101) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:17.212142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.213844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.221169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 273/537] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:17 2022] GET /identity/v3/role_assignments?user.id=e10e2b0b66754f128cddc94ca3daf101&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:17.306387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7ac61-33c1-4d75-a160-e632aab729a4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.354006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.354690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.354972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.355250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.355950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:17.356256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.357697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.376079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 265/538] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:17.382349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:17.485747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:17.497536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa67d1-9e1f-4dfc-af40-a6b552db300f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.542738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.543753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.544101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.544472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.552435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.553976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f37dee50-7379-4bfd-aae4-b16b15398adf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:17.569431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:17.570494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:17.571552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:17.572894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 274/539] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:17.579376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:17.644573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:17.695613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebcb38-f7b5-431b-8a8f-860ae617fca3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.727114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b32ed-8375-4690-80b2-1e92db87f302 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.727390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b32ed-8375-4690-80b2-1e92db87f302 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.727660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b32ed-8375-4690-80b2-1e92db87f302 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.729602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 275/540] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:17.734934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.735218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.735488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.739001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.739721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.740109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.740510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.741269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:17.741641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.743147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:17.756585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:17.756790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 266/541] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:17.757927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:17.769379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:17.841845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 276/542] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:17.937014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182ef13-32cb-4c0d-b962-04a39f69cf30 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:17.955665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:17.991820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:17.992540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:17.992814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:17.993080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:17.993807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=82a67430b66140e1b8427f2ffb627f42) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:17.994103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:17.995520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:18.003386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 267/543] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:17 2022] GET /identity/v3/role_assignments?user.id=82a67430b66140e1b8427f2ffb627f42&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:18.286141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:18.405384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:18.425678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb8c45-4d54-4235-b464-e4c6d75395db None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:18.479504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:18.480141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:18.480434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:18.480699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:18.494931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:18.499613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:18.501082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:18.501892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 277/544] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:18.507014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:18.635616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320bdb2-b6d3-4584-81c3-b37ef7da9a5e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:18.681851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:18.682461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:18.682735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:18.682999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:18.683648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:18.683943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:18.685314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:18.695402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 268/545] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:18.701181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:18.852593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b5339-7809-4633-9a55-ee5565830d2c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:18.911461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:18.912493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:18.912975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:18.913445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:18.929508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:18.931235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:18.933286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:18.947332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:18.948280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:18.949176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:18.949176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:18.949176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:18.951233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 278/546] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:18.957368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:19.020415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:19.086385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acc9c2-cf09-4a9f-a671-2a2487461ef4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:19.123660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:19.124254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.124556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.124827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.125973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:19.126289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:19.127642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec54fa-a7dc-45ec-992a-c56814d4335b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:19.144738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 269/547] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:19.153351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:19.308260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5c61a-04b1-4f62-862d-d21f6d351b67 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:19.329495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca843a-f1d7-4f07-9c66-aa8d5495ecd5 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.329842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca843a-f1d7-4f07-9c66-aa8d5495ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.330027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca843a-f1d7-4f07-9c66-aa8d5495ecd5 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.331524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 270/548] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:19.337429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.337716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.337990 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.352695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:19.353621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.354137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.354606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.362692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:19.364438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:19.374006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:19.375214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:19.377269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:19.378637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:19.379713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:19.382256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 279/549] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:19.392361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:19.451099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 271/550] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:19.476805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:19.542713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22aa65-1e7a-42ad-80a5-1200468f82ac None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:19.579795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:19.580884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.581398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.581869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.582852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:19.583361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:19.585070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:19.603828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 280/551] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:19.611822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:19.660811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:19.763634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac40fb-ab27-46b5-a1b9-f8c411744716 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:19.793443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:19.812025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:19.812672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.812985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.813255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.938674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd2276-1e78-43b6-b8f8-19b195e40ea3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:19.955906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:19.977334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:19.978979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016f916-cf1f-4022-ab5d-25cce0b2fc2e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:19.984540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74d241-beae-4499-854b-f14190299903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:19.986246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d241-beae-4499-854b-f14190299903 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:19.986731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d241-beae-4499-854b-f14190299903 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:19.987206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74d241-beae-4499-854b-f14190299903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:19.999710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74d241-beae-4499-854b-f14190299903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-aa74d241-beae-4499-854b-f14190299903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:20.005053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.006061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:20.006893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:20.009492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 281/552] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:20.010942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252ad79-570e-4792-a23b-1856a73865ac None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.011425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252ad79-570e-4792-a23b-1856a73865ac None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.011798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.012530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252ad79-570e-4792-a23b-1856a73865ac None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.015119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 282/553] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:20.020213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252ad79-570e-4792-a23b-1856a73865ac None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.104630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 272/554] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:16:20.104630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.104630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.104630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.122766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:20.123926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:20.128814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252ad79-570e-4792-a23b-1856a73865ac None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.165939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.166470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.166740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.167004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.167642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:20.167935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.169726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251214b2-ffe3-4c3c-b5f5-7aa0721740a3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.183139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 283/555] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:20.184435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853fcf-b24b-4a1a-9bf6-ff868a6212da None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.184724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853fcf-b24b-4a1a-9bf6-ff868a6212da None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.184990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f853fcf-b24b-4a1a-9bf6-ff868a6212da None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.186202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 284/556] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:20.186888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853fcf-b24b-4a1a-9bf6-ff868a6212da None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.196674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 273/557] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:20.197705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.228056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.267781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.302473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f853fcf-b24b-4a1a-9bf6-ff868a6212da None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.321903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6daddfd-9117-44e2-ae0c-df7ba02ed073 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.338231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.339002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.339343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.339678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.352031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.353649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.365278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.366117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.366539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.366951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-003b8e61-1f1d-40d7-b8e5-d3b84e37bf88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:20.368734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.369823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:20.370884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:20.370884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:20.370884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:20.375903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 285/558] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:20.375903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.375903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.375903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.387410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.389443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.393162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:20.394418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:20.405985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:20.408734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:20.408734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:20.408734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:20.410676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 274/559] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:20.411574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.488713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 286/560] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:20.490457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.512620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.535292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fdf5c-0078-4cf5-9edd-12db1f086509 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.546661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.577663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.579331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.579930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.580489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.581514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:20.582536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.583459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.601437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 275/561] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:20.601437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.629741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d342-8acb-466f-9c38-0cef13362800 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.680103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.680736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.681010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.681275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.681901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:20.682194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.683570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.700519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 287/562] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:20.701388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.723481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a8a5b-3e38-4c01-8858-62c52fdf06f7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.763214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.763760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.764044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.764337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.774176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:20.778938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.780031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 276/563] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:20.781189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.787329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.811705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7a29e-3cc5-4501-be84-235a4403cbc8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.811876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:20.843872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.845067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.845340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.845610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.852376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.853809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-2da64862-fb0e-4bf7-b326-245117934287 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:20.865380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.866266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:20.867179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 288/564] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:20.868200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a734d-8178-4294-a310-0273f3b69420 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.868200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a734d-8178-4294-a310-0273f3b69420 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.868200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a734d-8178-4294-a310-0273f3b69420 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.869331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 289/565] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:20.870018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a734d-8178-4294-a310-0273f3b69420 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.894035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a3e51-4207-49fa-9b5f-041e53062fe4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:20.927708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:20.928338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:20.928617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:20.928891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] PATH_INFO: `/v3/users/nova` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:20.935599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:20.937068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:20.948276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:20.949282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 277/566] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:20.950228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:20.974569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a734d-8178-4294-a310-0273f3b69420 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.007124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.007716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.007994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.008269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.008943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.009244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.010604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.012691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.020674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 290/567] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.021398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.041010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.050690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83eb9f-451d-4e63-8a9d-bf95fcde45df None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.083347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.083964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.084246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.084553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.094798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-fc045370-f9ab-4be1-ad30-27fa25713459 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:21.099899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:21.100987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:21.102070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 278/568] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:21.103166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.103166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.103166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.118790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbb3e7-60aa-4bee-b393-6a2112257a60 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.119771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:21.120971 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:21.151599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.152192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.152490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.152756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.153424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.153720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.155151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.173588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 291/569] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.173588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.186686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 279/570] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:21.187498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.230747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.274683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfbc73e-014b-490b-b17c-8db67d4f05bd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.281012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.302488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf640496-b9a4-4461-8a66-1ed7608657b4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.309495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.310101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.310375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.310640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.311345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.311642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.313629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.331318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 292/571] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.332217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.342437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.343429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.343702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.344323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.355517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.356976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:21.371236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:21.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:21.373120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:21.373120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.373120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:21.376596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 280/572] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:21.378103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.433584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba5a12c-449f-474f-a397-a7e6a21d2534 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.452006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.470591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.471306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.471585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.471852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.472522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.472821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.474188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.482556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 293/573] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.483308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.485557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b462701-4948-4c73-a742-bb430da55d91 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.515141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.521861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.522447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.522727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.523037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.523746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=40dc194ca31947339986b511f4271498) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.524052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.525560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.531226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 281/574] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/role_assignments?user.id=40dc194ca31947339986b511f4271498&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.531917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.586860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7ffd5-aa5e-4280-85b0-700d7fa2075e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.623649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.624316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.624607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.624860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.637402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.638964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.646801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85644eb1-f61f-4d84-b630-a16461509028 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:21.654750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:21.656206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:21.657884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:21.657884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.657884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:21.657884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 294/575] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:21.657884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.702993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.703649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.703923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.704877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.704877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:21.705575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.706984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.708498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.722263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 282/576] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:21.723469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.795464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:21.809521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e194412-590f-4c3a-9f58-6dad104f4fc2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.845236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda584-e972-47f4-a29c-e38a0dae73d0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:21.853846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.855014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.855374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.855716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.870723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:21.880622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:21.881553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:21.882386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:21.884729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 295/577] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:21.888425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.894085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:21.894826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:21.895197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:21.895610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:21.910320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:21.912835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:21.932274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:21.933455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:21.934529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:21.934529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:21.934529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:21.938630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 283/578] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:21.941010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:21.997824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.013057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a574990-ae71-4c96-899f-50ead8d1168e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.055478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.056170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.056650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.057011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.057851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.058237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.059701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.074972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d673f-7326-414a-ba0e-bfc59af4cad9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.078559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 296/579] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.079525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.086994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.119732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.120752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.121233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.121596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.129115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.130531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:22.141578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:22.142532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 284/580] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:22.144090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.206115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b5bf5-6cc8-4e87-86a9-be4d6fd29f45 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.243157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.243842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.244180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.244501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.245269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.245571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.247101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.248610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701eaf-6268-4d45-b006-5c9ec6dc014e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.255810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 297/581] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.261171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.276552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.283534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.284129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.284430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.284700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.285328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.285628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.286977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.303650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 285/582] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.303650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.331701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.369921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686050-e17b-4086-b448-2314bda57496 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.403704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.410101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.410101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.410101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.414338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.415835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.422980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415441a5-a0b7-420d-a5dc-570858c2d337 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:22.428540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:22.429580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 298/583] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:22.431075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.463754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.464695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.465129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.465405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.466146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.466452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.467866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.485605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 286/584] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.486393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.508406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.541275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e151f-e26b-49be-9ed5-d08acdfc06ce None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.575389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.576009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.576314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.576604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.578096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.589075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.590544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.595748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2096d8a-d8ef-4fbd-9968-29e1eb53ee20 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:22.605667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:22.606598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:22.607480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:22.607480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:22.607480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:22.607480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 299/585] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:22.607480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.635617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.636237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.636558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.636827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.645224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.646686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:22.661564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:22.662687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:22.663825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:22.670060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 287/586] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:22.670060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.720285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309da3-ad20-419a-9dcc-7a65d13ee88a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.743882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.756685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.757360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.757637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.758045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.758739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.759035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.760501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.772875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 300/587] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.773746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.780163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264511ba-16f8-4b3e-8be9-31a476159d72 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.817062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.818354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.818773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.819307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.831435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.833336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.841077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:22.850823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:22.852066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:22.853717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:22.853717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:22.853717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:22.853717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 288/588] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:22.853717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.884027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1547b-352f-450a-9913-800bd3407fb1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.919666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:22.920570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:22.920853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:22.921121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:22.921740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:22.922043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:22.923434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:22.935108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 301/589] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:22.938212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:22.962870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8bf3dd-e15b-4b70-ba74-55148a797866 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:22.995011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.002865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.003470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.003753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.004020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.004769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:23.005076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.006500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.029393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 289/590] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:23.029393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.052622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f57f4-ecdf-41fd-a547-c752b0659fe6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.086621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.089029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.089869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.090146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.090447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.105442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=6db6f737c3354ca692eb6ee91a496223, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.107133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.119922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 302/591] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:22 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:23.121080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.145406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0db2b-1040-4ce7-90fb-bf3ccda0c69f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.196348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.197402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.197859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.198312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.199160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:23.199847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.202090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.219962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 290/592] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:23.221232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.237769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.245770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88d3f47-2e4d-4b3a-969d-f551ab9fd4f8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.286526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.287281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.287633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.287971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.297023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.298535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:23.313650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:23.314629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 303/593] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:23.315606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.369644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884f58-1a26-4135-a42e-9c5f213a65ae None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.389765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.412559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.413612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.414116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.414577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.428041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.428567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7006b6e-8dc8-4829-bb5c-275a19679b39 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.430141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:23.446891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:23.447865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:23.448805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:23.448805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:23.448805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:23.450984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 291/594] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:23.455909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.472985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.473507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.473785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.474057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.481418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.482768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:23.494265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:23.495166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 304/595] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:23.496039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.518000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.585272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea5fa77-da4e-470b-bdd8-da6a92142f6a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.608907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40babf07-af6a-4e17-841a-b6fc86a56c0a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.627900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.628517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.628798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.629073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.629824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:23.630132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.636610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.655929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.656605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.657014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.657285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.657923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:23.658217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.659579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.660643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 292/596] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:23.661629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.676922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 305/597] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:23.677824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.697694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.759738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.783521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c832a1e3-2fee-4a3d-810b-10108e77d547 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.787106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af8636-a5b7-41d0-b8f6-0ab3e840ce83 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.817568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.818169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.818449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.818738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.819400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:23.819704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.821153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.823665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.824419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.824795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.825161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.838341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 293/598] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:23.838341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.841017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:23.842557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9596b310-435b-4289-9f52-6b642e2fcf58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:23.874119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:23.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:23.876961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:23.876961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:23.876961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:23.877249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 306/599] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:23.878924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c2bf-1262-4e9a-8490-999d42bf59ae None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.879011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c2bf-1262-4e9a-8490-999d42bf59ae None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.879214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74c2bf-1262-4e9a-8490-999d42bf59ae None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.885212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 307/600] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:23.885212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74c2bf-1262-4e9a-8490-999d42bf59ae None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:23.936062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:23.954782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff0427-4be7-4f7a-a5d0-3fa38ae75a7b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.991702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:23.992360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:23.992640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:23.992911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:23.994564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74c2bf-1262-4e9a-8490-999d42bf59ae None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:23.999680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.001752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:24.012849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:24.013735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:24.014646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:24.018537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 294/601] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:24.019213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.021501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.031716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.032583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.032719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.032959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/628750c611c24e8b9b4c0e66d2e3282e/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.050674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=628750c611c24e8b9b4c0e66d2e3282e, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.052453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebcce16-8b2c-4f0f-93b7-af05aa70f912 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.076339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 308/602] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:23 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/628750c611c24e8b9b4c0e66d2e3282e/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:24.076339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.076339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.076339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.100409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:24.102006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:24.131789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ae456-153c-45bb-8063-822a222a0a38 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.170424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.171129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.171473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.171816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.173088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:24.173389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.174799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.192505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 295/603] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:24.194002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.205756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.207581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 309/604] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:24.209018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.319077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f43792-825b-4502-bdc1-ecebe02647dd None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.322381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08546fa7-c028-4471-9c7a-f3db1717c11f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.331322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.359286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.360734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.361169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.361597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.363117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:24.363588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.365471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.369706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.370469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.370847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.371235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.378779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.379347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 296/605] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:24.380708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.385055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:24.393011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:24.394090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:24.395196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:24.397734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 310/606] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:24.406280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.461414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.504167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d783a-4fe1-44a7-8764-41d089b448d7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.518240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee5d79-a53a-4604-a040-8ad909656b15 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.546254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.547719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.548400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.548818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/e10e2b0b66754f128cddc94ca3daf101/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.557372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.558092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.558634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.559007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.559945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:24.560475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.562020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.566991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=e10e2b0b66754f128cddc94ca3daf101, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.568640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.576268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 311/607] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:24.582587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.587811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 297/608] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:24 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/e10e2b0b66754f128cddc94ca3daf101/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:24.615620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.632199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.704814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cbd65-ca88-45e6-a2ae-fe9cb8d3072a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.743645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.744351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.744633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.744906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/82a67430b66140e1b8427f2ffb627f42/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.761447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=82a67430b66140e1b8427f2ffb627f42, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:24.762578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e1bcc-2635-444f-b545-bbabdfcb2815 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:24.763009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:24.784689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.785371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 312/609] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:24 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/82a67430b66140e1b8427f2ffb627f42/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:24.804343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:24.805147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:24.805526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:24.805900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:24.818714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:24.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:24.825290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:24.826299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:24.832544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 298/610] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:24.835128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:24.939832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:24.967317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21901d-0a68-4930-8706-7e094d8d5be7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:25.005417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:25.006040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.006314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.006584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.007325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:25.007623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:25.009136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:25.021131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 313/611] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:25.026646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:25.131389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f506d-8e10-4a1b-92fb-fb5708d00a43 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:25.175950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:25.176941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.177411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.177858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.192307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:25.194001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:25.213730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:25.214729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:25.215650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:25.215650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:25.215650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:25.217532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 299/612] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:25.222779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:25.348326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d7c70-8113-4d6a-aeab-7ee5674f77c3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:25.391359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:25.392323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.392747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.393160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.393963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:25.394412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:25.395913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:25.403305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:25.409640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 314/613] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:25.414711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:25.450674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:25.549096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed187c56-dd74-4a1d-b72c-0982c44e28a2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:25.591361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:25.592352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.592858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.593329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.609337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:25.611695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2b423664-2a4d-4eab-84a6-a83d8d78580b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:25.626771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:25.627787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:25.628791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:25.631436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 300/614] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:25.636980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:25.745851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc7f0-1f92-47f8-b72b-3312614f949a None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.746176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc7f0-1f92-47f8-b72b-3312614f949a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.746399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cc7f0-1f92-47f8-b72b-3312614f949a None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.747816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 301/615] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:25.757811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.758096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.758350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.760732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a5f4d-0978-42c2-9abe-c2054d0b40ed None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:25.799728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:25.801999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:25.821547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:25.822557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:25.823054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:25.823554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:25.824519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:25.825038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:25.827938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:25.854720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 315/616] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:25.862413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:25.960765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 302/617] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:25.980361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.037376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f90b1-f883-4f54-bdba-b690aa2a5d78 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.087719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.088373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.088654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.088919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.099459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.100928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:26.119447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:26.120451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:26.121374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:26.121374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:26.121374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:26.125619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 316/618] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:26.130870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.150729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.250948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6c798-eb62-4064-96b7-4875b217bca1 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.257190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.294318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.294921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.295209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.295459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.296091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:26.296412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.297790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.326149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 303/619] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:26.327359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.427386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.449902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd8d85-f298-472f-becc-83b7dc57559a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.487523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.488147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.488461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.488732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.490230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f7aa2-c11d-4d41-ae6a-41d3e7822ba8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.495816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.498100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:26.510552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:26.511519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:26.512695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 304/620] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:26.514728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.520171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.601807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.602447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.602727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.603003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.613292 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:26.618508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:26.619436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:26.620486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 317/621] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:26.625721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.627035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a54ba-3e32-4116-aa1c-5e3e92f38df3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.668840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.669782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.670077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.670351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.671031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:26.671333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.673491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.682160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.689828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 305/622] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:26.698909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.771224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96a07e-f9af-42d1-adb5-b1a1a8efeb0c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.824525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.825587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.826167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.826663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.827576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:26.828128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.830233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.850203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 318/623] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:26.855653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:26.894046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82d46-1689-47fc-83f0-a80fc0533d7d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:26.906885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:26.930303 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:26.930965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.931248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.931523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/40dc194ca31947339986b511f4271498/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.946811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=40dc194ca31947339986b511f4271498, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:26.948209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9beb6-b66c-4f25-ac53-53a23dde9909 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:26.963195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 306/624] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:26 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/40dc194ca31947339986b511f4271498/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:26.965052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78a5f-70e9-4994-8070-15230032d165 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.965701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78a5f-70e9-4994-8070-15230032d165 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.966006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb78a5f-70e9-4994-8070-15230032d165 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.967906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 307/625] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:26.974716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:26.975476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:26.975852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:26.997681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1eaee-5583-4a73-9d0e-b3b9e974331d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.001842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:27.003019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:27.039032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.040103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.040627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.041118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.053913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:27.054117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.055551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:27.071982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:27.073315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:27.074594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:27.074594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:27.074594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:27.079584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 319/626] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:27.081029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.083394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 308/627] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:27.171371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.192752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d2eb1a-03fd-4e81-8e2a-8bea7e2961f4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.229291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.229828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.230106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.230377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.231023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:27.231325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:27.232853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:27.252133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 320/628] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:27.253500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.364792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee3d7a5-c4eb-41e2-a637-31dbe0a839ed None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.391325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.401678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.402318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.402597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.402866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.410750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:27.412212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:27.425044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:27.426163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:27.427148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 309/629] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:27.433300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.512909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.560819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092781f-dc7e-4b18-b325-e3eaa83dffbd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.590821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506065b-0e91-4193-8c09-908638b37f2e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.598077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.598726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.599005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.599277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.612234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-badeb9c4-7868-4885-8a9d-ac84de4f57f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:27.617358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:27.619258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 321/630] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:27.621143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaac15-c915-4c34-b91d-c136fd09e5fa None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.622815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaac15-c915-4c34-b91d-c136fd09e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.622815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbaac15-c915-4c34-b91d-c136fd09e5fa None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.622815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 322/631] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:27.627710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.628036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.628670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.635918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.636579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.637602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.638077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.638546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.640095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:27.641193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:27.643891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:27.658992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:27.660270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:27.677388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 310/632] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:27.678354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.749532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 323/633] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:27.750407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.792604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.803641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c767fb-9932-4cb5-874f-36824c89f943 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.842469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.843071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.843349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.843618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.844982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:27.845287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:27.846760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf6d31-431d-49a0-bc6e-cc0ebc4181c4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:27.857195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 311/634] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:27.860663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c806e-989c-43c0-a584-dcd806234845 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.860663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c806e-989c-43c0-a584-dcd806234845 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.860663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c806e-989c-43c0-a584-dcd806234845 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.865121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 312/635] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:27.865121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c806e-989c-43c0-a584-dcd806234845 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:27.891495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d30f5-8f32-4041-ad3d-7517101a967f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:27.932207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:27.939592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:27.940584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:27.941352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:27.942280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:27.984873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c806e-989c-43c0-a584-dcd806234845 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.017490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.021386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.023974 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.024980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.025347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.025709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.036643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.038227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-ba5e585d-99e9-4041-bc6c-b25133612186 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:28.054158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.055313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:28.056458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:28.056458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.056458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:28.060481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 313/636] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:28.060481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.060481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.060481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.077359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:28.079194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:28.083018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.138412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 324/637] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:28.140162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.149614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 314/638] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:28.150507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.179799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.268587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ff9f6-7495-4185-a729-1d0ebc7dfcda None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.271669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7304d2e-2c95-4510-add4-4189198f1482 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.306926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.307487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.307825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.308098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.310888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.311705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.312202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.312745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.313576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:28.314262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.316024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.318747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:28.325791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.326771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:28.327625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:28.330542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 325/639] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:28.331956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.333844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.342598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 315/640] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:28.344539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.435696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.445483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e80a1-b5be-4835-baaa-d5592e467d56 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.457653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c080e-7fa8-46cd-a2c0-7118a5870d7c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.482501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.483242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.483520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.483900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.491338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.491937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.492216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.492521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.493173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:28.493473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.494433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.494827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.495880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.503120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 316/641] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:28.503919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:28.515334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:28.517905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:28.517905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.517905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:28.517905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 326/642] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:28.517905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.572454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.614462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1cb269-b601-49f8-b090-a386e32f3750 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.621560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e154e-04d3-43c1-a87f-61cb9856e14d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.658767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.659683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.659957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.660230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.663040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.663661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.663932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.664197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.668361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.670080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.674249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:28.679408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.680627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:28.681488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:28.682126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.687951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 317/643] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:28.690225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.692521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 327/644] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:28.692521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.693345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.791601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3777b5-3b52-4ce6-84b1-90df5b4507b1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.795648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c90f90-62d2-4947-b1be-5ff10c236ad7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.819271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.826307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.827117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.827395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.827664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.828932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:28.829436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.831140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.832590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:28.833202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:28.833472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:28.833738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:28.848476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 328/645] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:28.848476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.850398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:28.851585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:28.870673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:28.871643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:28.873093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:28.873093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:28.873093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:28.876587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 318/646] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:28.877978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:28.917534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:28.962561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18637e3-9e94-498a-978e-f68639e8951b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:28.987739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535bc4c-a6f1-4621-96b0-b47ded25c88f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.000443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.001058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.001334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.001601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.002255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.002543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.003896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.011985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 329/647] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.012790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.029439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.030307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.030583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.030851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.032197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.032539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.034161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.057754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 319/648] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.057754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.089018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.117290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8350f9-a551-45a3-8da5-567e67ae076b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.144192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.153555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.154191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.154457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.154722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.162183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.163400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a871b64b-4343-410c-bea2-80ae444f740d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.163612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:29.174885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:29.175765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:29.176677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 330/649] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:29.177776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.204309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.204917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.205191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.205699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.206080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.206376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.207729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.219738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 320/650] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.220624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.287596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1cad7e-e387-4586-8741-07cc66e14ed6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.315159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.321234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.321960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.322128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.322400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.325948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e41609-e95a-4a4d-8fd5-3d8ad382621e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.334719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.336549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:29.355369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:29.356577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:29.357518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:29.357518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:29.357518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:29.358099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 331/651] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:29.358917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.365786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.366678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.367149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.367625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.399880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.442250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.443735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.474310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbdd02-6e39-4943-8eac-2f0550bd0ba4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.511756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.512390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.512677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.512960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.513631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.513934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.515361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.528621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 332/652] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.533699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.540139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.550056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 321/653] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:29.551163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.629130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.632689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e848fb0c-f218-4c9b-aef7-e1e91b5ceb9f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.648857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff791783-f0b6-43b0-afe0-d4b56efd44b9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.666488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.667109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.667385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.667655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.674507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.675928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.684354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.684953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.685225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.685490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.686109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.686403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.688285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c97e196-14d5-4194-a36c-a6fac7572048 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:29.688419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:29.689459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 333/654] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:29.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 322/655] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c87d9-a298-4855-96c8-e887e8632a05 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c87d9-a298-4855-96c8-e887e8632a05 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c87d9-a298-4855-96c8-e887e8632a05 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 323/656] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:29.706202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c87d9-a298-4855-96c8-e887e8632a05 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.763322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.800675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea10b43-8cbe-4a2e-80ad-9a4eefdb6bd5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.810519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c87d9-a298-4855-96c8-e887e8632a05 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:29.833936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.834530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.834798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.835065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.835759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:29.836057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.837498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.843063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 334/657] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:29.843841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.844418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:29.845045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:29.845312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:29.845577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:29.862417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:29.863898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:29.875765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:29.883342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:29.884403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:29.885329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:29.885329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:29.885329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:29.885329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 324/658] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:29.885329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:29.967481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c03fe1-bbf6-4e8c-af3e-2333daa36208 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.006085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.006700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.006973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.007066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3415a485-c084-4ea2-98b3-0cc2ef62ca1f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.007494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.007949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:30.008248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.009784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c8dd7-1828-4b22-b536-ef56891e123a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.010742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.028840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 335/659] 10.222.0.33 () {56 vars in 1139 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:30.029972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.030262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.030538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.047252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.048020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.048431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.048803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.050327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:30.051539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:30.056751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.058279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:30.074273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:30.075418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 325/660] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:30.076688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.129138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 336/661] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:30.129992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.141478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.196887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9ff0c9-4c1f-472b-9193-e6f811003655 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.232778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a0593-d17e-4462-a96e-0f24ec193b18 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.237041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.237650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.237926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.238197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.238836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:30.239136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.240533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.255349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 326/662] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:30.256198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.287045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.289096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.290101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.290385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.290657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.367265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3e7c6-bb87-4a7b-8258-a3692e1f0c6f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.383495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.385395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.401144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.404750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b181-039d-455c-ab29-7694458da67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.405367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b181-039d-455c-ab29-7694458da67f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.405744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b181-039d-455c-ab29-7694458da67f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.406642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290b181-039d-455c-ab29-7694458da67f None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.407892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290b181-039d-455c-ab29-7694458da67f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:30.408310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290b181-039d-455c-ab29-7694458da67f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.409866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290b181-039d-455c-ab29-7694458da67f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.428474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 327/663] 10.222.0.33 () {56 vars in 1137 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:30.429300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b181-039d-455c-ab29-7694458da67f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.508555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 337/664] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:30.508555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.515951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.537257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290b181-039d-455c-ab29-7694458da67f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.573577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.574211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.574486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.574753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.583293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.585102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:30.597019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:30.598117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:30.599047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 328/665] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:30.599534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.612457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b8917-0021-4275-86f9-86119ef499a7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.623678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.648068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.648703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.648977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.649239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.712743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c0487-681f-40a8-8e03-2e468c574ce8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.717876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.719876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.750402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.751441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.751714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.751989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.764027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:30.769386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:30.770311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 329/666] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:30.771202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.779196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.837058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 338/667] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:30.837058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.850094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:30.887624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b77b9-cef1-448e-aaba-e75c0d917435 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.919567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.920179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.920492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.920757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.931645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.933093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:30.947600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:30.948581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:30.949532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:30.949532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:30.949532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:30.949532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 330/668] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:30.949532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:30.950241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc17b3-da84-429a-ac4b-7f160beba5c5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:30.986495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:30.987102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:30.987277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:30.987607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:30.988205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:30.989144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:30.989940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.008133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 339/669] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.008133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.016504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.067558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463dba5-a0b0-4aa1-b919-f436c503d11c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.078479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.102800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.103401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.103673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.103943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.104620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.104920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.106278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.113612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83cf878-796d-44cb-8f2d-a5b4508fef7d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.119467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 331/670] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.119467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.154324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.155049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.155395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.155739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.167022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.168564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:31.183124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:31.184249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:31.185351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:31.185351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:31.185351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:31.190673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 340/671] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:31.190673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.228053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bc3ba-6619-4728-952f-41777958d95e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.244525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.263004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.263612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.263887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.264157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.264808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.265105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.266490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.278171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 332/672] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.283855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.295802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4d0ce-b1eb-4637-bf74-4ab0d15d736c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.317544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.332109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.332982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.333461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.333799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.334572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=628750c611c24e8b9b4c0e66d2e3282e) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.334877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.336283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.352325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 341/673] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/role_assignments?user.id=628750c611c24e8b9b4c0e66d2e3282e&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.352325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.410528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a542f-0660-4843-842f-d12011a084f3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.454542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.455224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.455496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.455760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.470428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.472239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.485502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140852b2-0784-4e12-92f5-b03e0272d301 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:31.491569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:31.492643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:31.493571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:31.493571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:31.493571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:31.498409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 333/674] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:31.499319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.539878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.540534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.540810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.541682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.541682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.541996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.543361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.543541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.560131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 342/675] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.560368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.618239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.638731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad78b77-0011-418b-9730-d6d14407e648 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.685384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124208e-a0cb-4e69-a611-b538a6cbe953 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.687270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.687903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.688179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.689103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.695674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.697246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:31.713407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:31.714534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:31.715651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:31.720948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 334/676] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:31.720948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.732258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.733393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.733820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.734267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.735233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.736043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.737961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.754178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 343/677] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.756229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.816927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.829671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f7ba09-196d-4dde-864c-75f7dec5cd5a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.864898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.865811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.866233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.866672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.877080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.878711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.879199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f35b7-c705-4c2e-a4fa-0ccd73f39d9e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:31.888631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:31.892160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:31.893284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:31.894353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:31.896602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 335/678] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:31.897557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:31.918957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:31.919686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:31.920030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:31.920411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:31.921493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:31.921661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:31.923168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:31.941384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 344/679] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:31.941384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.011842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4b25f-b915-44f8-a83d-c2452e2562aa None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.051862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.053290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.053748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.054196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.058439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2fa620-cf8c-48de-8844-31a9b7c5bcb6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.063341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.065017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:32.077059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:32.078653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:32.079602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:32.081822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 336/680] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:32.082812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.094262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.103638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.104577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.105064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.105561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.106556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:32.107059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.108815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.121200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 345/681] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:32.122256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.152320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.190150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687609b2-48e2-469d-8322-be99cefbb9f4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.225062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.226223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.226603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.227185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.228238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=82a67430b66140e1b8427f2ffb627f42) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:32.228581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.229741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc65cf5-5b97-47a3-9534-6f4356f07cf7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.230003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b026dbf-13ed-4e41-9bea-0ffabdc17d72 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.238269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 337/682] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/role_assignments?user.id=82a67430b66140e1b8427f2ffb627f42&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:32.239778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5537-16aa-4357-8d0f-7ac157837c26 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.240090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5537-16aa-4357-8d0f-7ac157837c26 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.240392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f5537-16aa-4357-8d0f-7ac157837c26 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.242069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 338/683] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:32.242998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f5537-16aa-4357-8d0f-7ac157837c26 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.279855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.282227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.282821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.283417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.284399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:32.284935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.286715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a34e60d-6d9e-4438-bac3-c8e150f401b4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.310155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 346/684] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:32.311764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.312878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.313370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.333598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:32.335268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:32.361071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.402485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f5537-16aa-4357-8d0f-7ac157837c26 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.424541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 347/685] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:32.425757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.450846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.453676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.454414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.454810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.455163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.455999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=e10e2b0b66754f128cddc94ca3daf101) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:32.456442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.457967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.466719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 339/686] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/role_assignments?user.id=e10e2b0b66754f128cddc94ca3daf101&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:32.557701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a26ff-b323-4531-96ef-ab9a4603ef84 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.595579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.596486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.596931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.597403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.674116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:32.675794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:32.690896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.716623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.803573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 348/687] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:32.808419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:32.852267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050eca19-0588-4589-a711-76899f6bead3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:32.898123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:32.898739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:32.899012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:32.899280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:32.910305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:32.920536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:32.929562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:32.930519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:32.931382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:32.932883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 340/688] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:32.936192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.014224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597de4d-f150-40c8-a59b-f3cb4313272f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.040277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e3-3a80-4d83-993e-b73323b8bd43 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.050067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.075261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.075883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.076133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.076458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.077117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:33.077409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.079311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.089509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 341/689] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:33.098052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.101205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.102012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.102333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.102482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.125447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.128579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:33.157437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:33.158439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:33.159329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:33.159329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:33.159329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:33.159329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 349/690] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:33.168050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.183508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.229853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9233bb-1fac-4354-b747-7c45fb64c87a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.266947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.267576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.267852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.268121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.279268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.280750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:33.296281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:33.297527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:33.298506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:33.298506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:33.298506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:33.298506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 342/691] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:33.303425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.372194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99f8d-6090-49f0-9cd6-bbd1ede103e8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.432745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.433456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.433731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.434004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.434640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:33.434938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.436561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.454987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 350/692] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:33.460338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.463326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf12c9-9513-4a39-ab6b-1335db8e4c23 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.488961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.489802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.540165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.540985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.542104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.542104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.542104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:33.542316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.543736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.569288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417ac3f-c872-486a-bf81-c3fae1bfa127 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.578932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 343/693] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:33.589344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.611562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.612201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.612508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.612782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.619753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.621335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:33.633219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:33.634103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:33.635005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 351/694] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:33.640203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.740722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.753259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66581850-0f71-4a62-8f61-0329ca513950 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.817666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.818381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.818661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.818925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.819598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:33.819893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.821614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.832251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525963a5-b7ac-462b-8d01-dd1aed9baa9f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:33.834118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 352/695] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:33.839942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.896702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:33.897615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.898590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.899089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.909354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:33.911056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-28c6974d-f7dc-4acd-94ab-abadd42629c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:33.928118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:33.929117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:33.930103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:33.933034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 344/696] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:33.938050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65e40-909b-4488-a99f-f5e4beb106d3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:33.938050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65e40-909b-4488-a99f-f5e4beb106d3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:33.938050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac65e40-909b-4488-a99f-f5e4beb106d3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:33.940584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 345/697] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:33.941354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac65e40-909b-4488-a99f-f5e4beb106d3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:33.975737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:33.977628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1fd62-b171-467f-beb2-356f27facd3d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.025256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.026283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.026806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.027300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.028211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=40dc194ca31947339986b511f4271498) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:34.028736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:34.031440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2d3f8-1f66-430d-8641-3a95f2afe99e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:34.048621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 353/698] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/role_assignments?user.id=40dc194ca31947339986b511f4271498&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:34.048621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b558-c242-4299-b0a7-352358495e15 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.048621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b558-c242-4299-b0a7-352358495e15 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.048621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217b558-c242-4299-b0a7-352358495e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.077315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0217b558-c242-4299-b0a7-352358495e15 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:34.078585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0217b558-c242-4299-b0a7-352358495e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:34.092692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:34.096674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac65e40-909b-4488-a99f-f5e4beb106d3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.179629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.180176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.180485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.180823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.181560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:34.181856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:34.183304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:34.204556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 354/699] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:34.224091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 346/700] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:34.229883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217b558-c242-4299-b0a7-352358495e15 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:34.350726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217b558-c242-4299-b0a7-352358495e15 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.386213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.387161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.387661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.388130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.399492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:34.460803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:34.462564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:34.552272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:34.574541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 355/701] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 12:16:34.580829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:34.626677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:34.670119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:34.685716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642df2fb-4959-4061-8a2b-6b21ee939736 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.691948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f65aa-2630-423a-8a06-4a762d4a4048 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.729939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.730680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.730793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.731023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.736685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.737631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:34.738694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:34.739168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:34.741799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:34.743275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:34.753805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:34.755617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:34.759135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:34.760192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:34.761125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:34.761125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:34.761125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:34.761125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 347/702] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:34.766896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:34.786466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:34.787486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:34.788416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:34.788416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:34.788416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:34.788416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 356/703] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:34.795069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:34.947416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efeccb-4620-4172-adab-2bf195fecd44 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.956897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c834e12-410c-4679-95a3-ea80c7728712 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:34.984801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:34.997936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:34.999232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.000269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.000872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.002001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:35.002463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.003764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.004258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b7cef-e946-4d48-887a-82f7eaa9360f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.004745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.005188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.005607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.006497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:35.006943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.008581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4eafc5-03e8-45a3-9dbf-ba5ad2b7f0df None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.018854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:35.019497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 357/704] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:35.020999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288fb4d-10a3-4179-802b-3173b6336ad4 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.021531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288fb4d-10a3-4179-802b-3173b6336ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.022000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1288fb4d-10a3-4179-802b-3173b6336ad4 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.023880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 358/705] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:35.028795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 348/706] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:35.028956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288fb4d-10a3-4179-802b-3173b6336ad4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.032575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e248b-e690-421e-878e-ff435f333f33 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.032868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e248b-e690-421e-878e-ff435f333f33 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.033143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e248b-e690-421e-878e-ff435f333f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.052099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-171e248b-e690-421e-878e-ff435f333f33 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:35.053336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-171e248b-e690-421e-878e-ff435f333f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:35.125916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 349/707] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:35.126743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e248b-e690-421e-878e-ff435f333f33 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.151844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1288fb4d-10a3-4179-802b-3173b6336ad4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.200096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.201736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.202250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.202995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.213152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.215271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-cf1f7494-fc59-495c-a4b6-4090ae9a55f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:35.231420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:35.232386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:35.233276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:35.236484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 359/708] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:35.237872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62e67f-15a8-4236-b1a5-7db5e89ad7fb None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.238586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62e67f-15a8-4236-b1a5-7db5e89ad7fb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.239188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a62e67f-15a8-4236-b1a5-7db5e89ad7fb None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.241720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 360/709] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:35.243349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62e67f-15a8-4236-b1a5-7db5e89ad7fb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.249287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e248b-e690-421e-878e-ff435f333f33 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.264166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:35.289264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:35.298352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.299529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.300006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.300822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.302133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.304267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d77fff-2e85-41e9-8f19-1759299acd50 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.336465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 350/710] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:16:35 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:35.338189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.338712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.339180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.364187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:35.366144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:35.390173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a62e67f-15a8-4236-b1a5-7db5e89ad7fb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.430209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.430838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.431168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.431460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.432192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:35.432528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.434075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.449032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 361/711] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:35.449947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.461039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 351/712] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:35.462354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.587373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a508a6b-7562-4f3f-a287-8b45e9c33e4c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.591132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:35.611861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:35.626246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2979e-2804-4383-ab7a-4397ba8b8066 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.633637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.634230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.634505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.634769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.635477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:35.635778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.637404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.644366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 352/713] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:35.678879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.679534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.679806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.680070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] PATH_INFO: `/v3/services/network` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.680679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.682123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:35.696763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:35.697855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:35.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 362/714] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:35.705675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.818559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.833445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ff913-22ab-4571-8448-09bd7a089b1d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:35.878601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:35.879210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:35.879484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:35.879752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:35.881006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] RBAC: Adding query filter params (name=network) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:35.881006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:35.882065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:35.891772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 353/715] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:35.893110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:35.946029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:36.001039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f233-b87d-45b8-b2b7-400443904849 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:36.006072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b962a2-ee7e-4972-8310-32936216ec7b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:36.038097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:36.039183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.039679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.040156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:36.041010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] RBAC: Adding query filter params (type=network) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:36.041607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:36.043178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:36.049697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 354/716] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:36.089890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:36.090545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.090819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.091088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:36.091668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:36.093087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:36.103464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:36.113248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:36.114337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:36.115176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 363/717] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 22 12:16:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:36.123634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:36.299900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e8ee1-bbce-45d8-b1f6-1fa2c9875926 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:36.310842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:36.340848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:36.341460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.341722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.341989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:36.342605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:36.342898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:36.344390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b8f0e-723f-4462-a748-0f32aa87a430 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:36.349063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 355/718] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:36.354716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:36.460694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ec0191-1589-4142-ac4d-0a5481aebe79 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:36.495818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:36.496451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.496723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.496989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:36.497612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:36.497905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:36.499266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e21fe9-45ed-4f8a-a33c-d9bcb09182ad None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:36.516273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 364/719] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:36.974322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f856a8-32eb-4696-b591-3d7371dd48ac None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.974679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f856a8-32eb-4696-b591-3d7371dd48ac None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.974860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f856a8-32eb-4696-b591-3d7371dd48ac None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:36.976739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 356/720] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:36.982493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:36.982780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:36.983058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:37.010178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:37.011692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:37.085700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:37.098721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 365/721] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:37.517221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f856a8-32eb-4696-b591-3d7371dd48ac None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:37.632135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f856a8-32eb-4696-b591-3d7371dd48ac None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:37.667356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:37.667981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:37.668252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:37.668543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:37.683003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:37.687919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:37.688929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:37.689793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 357/722] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:37.695000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:37.797005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12a6a4-961b-4473-aed8-dda99c9d649b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:37.847144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:37.847801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:37.848077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:37.848361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:37.849024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:37.849320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:37.850943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:37.863659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 366/723] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:37.869492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:37.905743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:37.976444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5d940-7da2-4f20-8885-7a37561ccc76 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.021642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:38.022335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.022617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.022892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.033257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:38.034712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:38.052801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:38.053737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:38.054726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:38.054726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:38.054726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:38.054726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 358/724] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:38.059554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:38.135361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:38.189191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13e415-5845-431c-ac65-9ec9aa923584 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.236124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:38.237169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.237680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.238195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.239042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:38.239548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:38.241145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9aa7e-580a-44a7-9219-5deb102f0587 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:38.261540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 367/725] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:38.269724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:38.338133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2cd3-c239-4a06-9626-dd111f9784b9 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.338473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2cd3-c239-4a06-9626-dd111f9784b9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.338705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3b2cd3-c239-4a06-9626-dd111f9784b9 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.340465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 368/726] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:38.346824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dc7-8276-45d6-b9ac-db66ff2fafc0 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.346824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dc7-8276-45d6-b9ac-db66ff2fafc0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.349583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dc7-8276-45d6-b9ac-db66ff2fafc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.369981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4f3a6dc7-8276-45d6-b9ac-db66ff2fafc0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:38.371167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4f3a6dc7-8276-45d6-b9ac-db66ff2fafc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:38.431470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:38.461665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e686d28-1b95-48b0-917a-9b6d986fc95c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.463240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 369/727] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 12:16:38.466358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215857e9-b757-4280-a278-e5a84c390b93 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.466855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215857e9-b757-4280-a278-e5a84c390b93 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.467290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215857e9-b757-4280-a278-e5a84c390b93 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.469157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 370/728] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:38.477279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.477546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.477803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.519223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:38.522947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:38.549419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980351c6-1d28-4561-854d-215e77018414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:38.550145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980351c6-1d28-4561-854d-215e77018414 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.550357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980351c6-1d28-4561-854d-215e77018414 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.550546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980351c6-1d28-4561-854d-215e77018414 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.563653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980351c6-1d28-4561-854d-215e77018414 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:38.566395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980351c6-1d28-4561-854d-215e77018414 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-980351c6-1d28-4561-854d-215e77018414 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:38.583755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:38.586079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:38.588500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:38.593933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 359/729] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:38.596877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:38.597105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980351c6-1d28-4561-854d-215e77018414 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:38.645955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 371/730] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:38.771451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:38.773022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980351c6-1d28-4561-854d-215e77018414 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.820718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:38.821335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.821613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.821880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:38.822595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:38.822890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:38.824319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:38.843438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 360/731] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:38.848748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:38.851700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:38.961560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33097a0c-cf82-4a1f-8fe7-d1f71a2b4dcf None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.995066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5895a5-75ba-4f93-b679-60d2a7ef3dc2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:38.997736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:38.998375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:38.998654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:38.998924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.010326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:39.015196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:39.016126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:39.016994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:39.021272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 372/732] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:39.024315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.036567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.037175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.037447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.037970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.059649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.118378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.120045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09e9ee-7adb-4590-8bf2-67b14b604dd3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.122094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.126973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d827178-c043-423f-bed5-0cc86cb18cb0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.162976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.163597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.163868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.164137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.175714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:39.180691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:39.181645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 373/733] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:39.183186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 361/734] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118158c-0b7a-4347-9958-1506faa4bfe6 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118158c-0b7a-4347-9958-1506faa4bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118158c-0b7a-4347-9958-1506faa4bfe6 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 362/735] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:39.231443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118158c-0b7a-4347-9958-1506faa4bfe6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.294569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.303244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43c93c-fd9f-4dc3-9ed2-a8b56e595065 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.339623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.340352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.340532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118158c-0b7a-4347-9958-1506faa4bfe6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.340635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.340875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.341518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:39.341812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.342322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.343703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.352383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 374/736] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:39.353030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.391471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.391997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.392272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.392581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.393239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:39.393538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.394891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2442c486-0bb5-47dc-a036-50076b57eaef None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.402769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 363/737] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:39.403898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.404179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.404480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.422228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:39.423394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:39.454331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c1747-ae0e-403f-8934-2904e9c0a783 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.487653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.488073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 364/738] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:39.488277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.488581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.488852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.488949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.510864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.512437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-20448315-c483-4196-acf2-a263eb5303f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:39.526890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:39.527883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 375/739] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a00e5-efd3-46ab-836d-cbb31c00aecb None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a00e5-efd3-46ab-836d-cbb31c00aecb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a00e5-efd3-46ab-836d-cbb31c00aecb None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 376/740] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:39.534214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a00e5-efd3-46ab-836d-cbb31c00aecb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.545556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.592607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.600449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a4137-4017-4e34-bb71-ae252ce11764 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.637083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.637706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.637988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.638265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.643002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a00e5-efd3-46ab-836d-cbb31c00aecb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.649521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.650966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:39.667058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:39.668008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:39.668958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:39.668958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.668958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:39.668958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 365/741] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:39.668958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.682813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.683783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.684271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.684859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.698665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.700152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d2dfc322-5f3a-4a74-a550-0c0b9b71b033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:39.716677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:39.717717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 377/742] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684253-fb08-4676-827c-780d3253a553 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684253-fb08-4676-827c-780d3253a553 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.724538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23684253-fb08-4676-827c-780d3253a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.741238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-23684253-fb08-4676-827c-780d3253a553 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:39.742454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-23684253-fb08-4676-827c-780d3253a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:39.801660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.812648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 378/743] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 12:16:39.813451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684253-fb08-4676-827c-780d3253a553 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.898701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36752ef3-a598-400f-a644-a6b7c17f5193 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.932695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23684253-fb08-4676-827c-780d3253a553 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:39.934114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.934709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.934984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.935876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.935876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:39.936171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.937561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.948869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 366/744] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:39.949494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:39.958625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:39.974812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:39.975420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:39.975954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:39.976226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:39.976886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:39.977205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:39.978550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:39.994747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 379/745] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:39.994747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.046303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.060130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62388266-1dd3-402e-aede-efd8c9e9c96e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.095656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.096249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.096411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e0eeb-ac00-4290-9771-b0e1578c0c87 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.096553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.096824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.097451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:40.097750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.099102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.110073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 367/746] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:40.110845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.132198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.132828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.133101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.133379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.134011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.135353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.151623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 380/747] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:16:39 2022] POST /identity/v3/services => generated 240 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:40.152670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.183064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.225436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a779-ac50-419e-a7a6-83d232be1f29 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.277588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e592c24-6916-4aa9-a89e-9aa62203c479 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.280516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.281156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.281437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.281704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.288510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.289228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.291135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:40.305741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:40.306770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:40.307670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:40.312485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 368/748] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:40.313260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.321826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.322467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.323071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.323071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.332076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.333601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:40.345414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:40.346372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 381/749] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:40.347307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.425120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7b4d-e2b5-48f6-af3a-2cd1fd013bee None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.451262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.461644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5eedd-d977-403c-b944-3957fcb8dee2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.464177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.464806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.465078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.465373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.466036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.467387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.483139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 369/750] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:16:40 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:40.484289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.522171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.523928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.524442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.524929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.536484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.539172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:40.557096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:40.558139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:40.559087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:40.567787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 382/751] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:40.567787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.604247 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.617174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50049bd-c281-47bc-bf20-6fa07cdffc5c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.672721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.673735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.674178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.674433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.675174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:40.675590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.677876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.692193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 370/752] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:40.692876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.702002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dcb34-cc97-4309-8b5e-c33cfa562fe2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.742238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.742871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.743147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.743419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.744134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:40.744469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.745898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.752797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.766022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 383/753] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:40.766864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.823580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370cd99-eace-45c8-8355-b0370287323b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.861512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:40.872092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.872738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.873013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.873279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.874004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:40.874793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.875789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.889279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31745708-3b8a-4f5b-9805-89a5401dff40 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:40.896319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 371/754] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:40.897308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:40.930160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:40.931106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:40.931644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:40.932202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:40.933284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=40dc194ca31947339986b511f4271498) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:40.933802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:40.935422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:40.942095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 384/755] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/role_assignments?user.id=40dc194ca31947339986b511f4271498&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:40.943229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.105711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:41.114503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c3f8a-75a0-4d6c-b170-6cc21a304b9a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:41.185492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:41.186021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:41.186293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:41.186558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:41.200735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef073ff-65c4-4cfe-8db2-fd19e012bc82 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:41.244789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:41.245786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:41.246259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:41.246723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:41.258551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:41.260489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:41.269610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:41.279355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:41.281211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:41.282096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:41.282096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:41.282096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:41.284158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 385/756] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:41.289861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.335325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:41.337147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:41.445245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dade3e2-dda6-4801-b3cc-6b4c4cc29ab5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:41.487914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:41.489336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:41.489823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:41.490287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:41.491115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:41.491654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:41.493237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:41.507521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 386/757] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:41.513610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.550244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:41.557301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 372/758] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:16:41.559448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.648693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:41.711019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488ae69-9c4f-4c20-9443-5f1b988a7b38 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:41.725435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee8033-d5cd-40d3-923e-56c6e0aa4f54 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:41.777547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:41.779048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:41.779362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:41.779633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:41.792217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:41.792876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:41.793145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:41.793408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:41.799730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:41.801209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:41.804318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:41.805732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:41.825252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:41.826169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:41.827087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 387/759] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:41.832412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:41.841390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:41.842328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:41.843296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:41.843296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:41.843296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:41.846032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 373/760] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:41.852557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:41.940953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:42.024025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a27cc-c543-4b36-bd7f-629fc5cceb1e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.028729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c846326-890b-45f0-b8f1-63140bd1fda6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.057832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:42.086113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.086709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.086980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.087242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.087906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:42.088200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.089645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.095332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.095919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.096189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.096480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.097104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:42.097400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.098733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.117191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 388/761] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:42.120696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 374/762] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:42.122295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:42.125813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:42.299616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ffb659-1e2f-403b-ac84-ca3c4a2eddb0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.304835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c68bd70-f5c6-451c-b106-c8c2f8f657c3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.318906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:42.345441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.346097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.346590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.346860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.354989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.356449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:42.369437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:42.370351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:42.371247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 375/763] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:42.372938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.373601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.373868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.375805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.376595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:42.378187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=6db6f737c3354ca692eb6ee91a496223, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.379598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.409285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 389/764] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:42 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:42.494510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ff3eb-8785-436a-8249-e9c5cded3a1b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.532402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.533457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.533938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.534408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.535335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:42.535851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.537640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85774190-931a-40fc-907a-26a01e6bc4ec None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.553237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 376/765] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:42.559450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:42.627853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:42.672616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26711024-3066-4526-9cd2-fd129f88cf9f None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.672922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26711024-3066-4526-9cd2-fd129f88cf9f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.673185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26711024-3066-4526-9cd2-fd129f88cf9f None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.677042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 377/766] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:42.682702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdb1ab-c9ca-476e-8ea9-3141e0e7adad None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.683000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdb1ab-c9ca-476e-8ea9-3141e0e7adad None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.683259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cdb1ab-c9ca-476e-8ea9-3141e0e7adad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.714214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-81cdb1ab-c9ca-476e-8ea9-3141e0e7adad None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:42.715418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-81cdb1ab-c9ca-476e-8ea9-3141e0e7adad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:42.732733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ac915-3fcd-400f-8d2b-7da2c0f6eb5c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:42.759916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:42.772284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:42.772960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.773230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.773497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.774251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=2fd208a28c9d41288c294cdf01bf6a31) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:42.774553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:42.775971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637452b-3697-4ad3-a124-718ac2b15e6f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:42.782425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 390/767] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:42 2022] GET /identity/v3/role_assignments?user.id=2fd208a28c9d41288c294cdf01bf6a31&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:42.816046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 378/768] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:42.916835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954cdd7-bc34-4844-a661-f7620a3c1fa3 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.917134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954cdd7-bc34-4844-a661-f7620a3c1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.917535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954cdd7-bc34-4844-a661-f7620a3c1fa3 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.919245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 391/769] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:42.925743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:42.926105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:42.926311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:42.957424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:42.958970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:43.054908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 379/770] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:43.137941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954cdd7-bc34-4844-a661-f7620a3c1fa3 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.269036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954cdd7-bc34-4844-a661-f7620a3c1fa3 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.270672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:43.314740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.315419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.315695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.315955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] PATH_INFO: `/v3/services/network` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.316548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.317900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:43.329506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:43.330528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:43.331366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.335452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 392/771] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:43.336878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.385723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.470517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:43.480250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d9a9e-fe07-4769-9a3c-fbb729c3fdc8 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.521420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.522180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.522591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.522874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.523612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] RBAC: Adding query filter params (name=network) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:43.523923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.525432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c751938-0b6a-45f7-8241-713a35da2c4c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.530883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 380/772] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:43.532848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c63f9-d3a1-4703-bd87-667717899222 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.533131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c63f9-d3a1-4703-bd87-667717899222 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.533520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c63f9-d3a1-4703-bd87-667717899222 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.535089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 381/773] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:43.536581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c63f9-d3a1-4703-bd87-667717899222 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.544027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10066a-21df-4c87-8633-0a8ae4453c20 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.581795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.582420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.582691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.583019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.583694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.585160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-781c1dd9-453c-4f4f-a694-dd6cb1f15a8d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:43.594963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:43.595936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:43.596822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 393/774] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:43.597614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.598198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.598198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.617982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:43.619355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:43.646486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:43.657223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c63f9-d3a1-4703-bd87-667717899222 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.697157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.697774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.698047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.698315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.698503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 394/775] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:43.699000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:43.699237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.699413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.701316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.708746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 382/776] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:43.717906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.768772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:43.848953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fbc06-f9dd-4a4c-aa8e-3821ad540add None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.909284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63144746-82c9-4b96-8b0e-336f5c8a6cc1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:43.949502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.950100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.950372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.950632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.951314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] RBAC: Adding query filter params (interface=public, service_id=474545d6911c45d18b8bbe1e4c9d0792, region_id=RegionOne) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:43.951609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.953616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.957540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:43.958148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:43.958414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:43.958675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:43.959286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:43.959578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:43.960934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:43.969987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 395/777] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:43.974480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 383/778] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/endpoints?service_id=474545d6911c45d18b8bbe1e4c9d0792&interface=public®ion_id=RegionOne => generated 194 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:43.977062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:43.994075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.029704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.095667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.115086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec1259-75e4-49b0-bce1-eade07476953 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.137576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988467a-008a-4e12-88e4-bc7232f9851f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.157858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.158462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.158738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.159005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.159622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.161017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.166873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 384/779] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:44.167575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.183228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.184771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.185281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.185801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.187208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:44.187741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.189473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.195652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 396/780] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:44.201732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.287568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218213-b65f-47ca-aeb5-25aade57dd71 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.300745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.314562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdee1df-33c3-4ab5-8364-042debbb37bf None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.327444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.328081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.328578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.328716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.351288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:44.362890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.364107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.364614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.365127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.367059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] RBAC: Adding query filter params (interface=public, service_id=4ce00ecf4bbe453ba4314bb255b029d8, region_id=RegionOne) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:44.368412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:44.369646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:44.371389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:44.372248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 385/781] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:44.380365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.381350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.390333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 397/782] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/endpoints?service_id=4ce00ecf4bbe453ba4314bb255b029d8&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:44.396569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.404125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.518789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d75ae-8b0e-4035-b83e-2b6589f02a7e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.537243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbcc3e-2a8e-4770-b5ea-ef4f152266f5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.566828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.567585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.567872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.568569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.569335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.571094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.579752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 398/783] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:44.586082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.586713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.586989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.587259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.587945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:44.588246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.589705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.600351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 386/784] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:44.606378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.628674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.759600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0649c3-e418-4908-bb7b-ec8e6634f1d2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:44.814143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:44.815159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:44.815632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:44.816093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:44.832580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:44.834443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:44.854811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:44.855833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:44.856758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:44.856758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:44.856758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:44.859583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 399/785] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:44.865540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:44.949653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:44.984351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a13a9-1c16-4e31-9069-fbb54b710798 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.022686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.023304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.023582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.023854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.024530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:45.024834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:45.026251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375edda-6830-48f2-b081-35f790f82e73 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:45.042316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 387/786] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:45.048348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.129739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:45.129915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77863ce-c499-4b9e-ab3f-fe9c95db30eb None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.130169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77863ce-c499-4b9e-ab3f-fe9c95db30eb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.130443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77863ce-c499-4b9e-ab3f-fe9c95db30eb None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.132021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 388/787] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:45.144554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.144836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.145110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.169074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb17de0-7a20-4c08-86b5-f6a785ae4af5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.178235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:45.179876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:45.228794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.229425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.230219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.230496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.244533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:45.245977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:45.258496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:45.259495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:45.260460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 400/788] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:45.265481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.336985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 389/789] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Sep 22 12:16:45.380904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff154c-bdd2-4a95-b7fa-2c4e5e251ecc None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.407417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:45.416096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.416824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.417349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.417780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.418822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:45.419125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:45.420577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:45.446720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 401/790] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:45.452885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.517687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:45.560491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab16f0-38f2-4661-8e21-5cf855b56739 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.593464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.594077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.594350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.594614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.604796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:45.606214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:45.621353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:45.622324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:45.623213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:45.623213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:45.623213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:45.623213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 390/791] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:45.628224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.687832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.766170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f9187-8464-4b5d-af81-d2a5aace3002 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.798187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84cc04-7afd-4e88-8dc7-450f01e49047 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.802496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.803100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.803374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.803642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.804360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:45.804739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:45.806152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614b266-6c52-4ce9-83a4-e29cbb684f1b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:45.814040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:45.819679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 402/792] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:45.821068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7011-4819-4d65-9a0f-af4c5d6db022 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.821349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7011-4819-4d65-9a0f-af4c5d6db022 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.821620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c7011-4819-4d65-9a0f-af4c5d6db022 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.823949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 403/793] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:45.824683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c7011-4819-4d65-9a0f-af4c5d6db022 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:45.838144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:45.839094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.839573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.840044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.851019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e14db878-2261-4fe7-92bc-4567ac81ddef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:45.856487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:45.857408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:45.858810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:45.861429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 391/794] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:45.862838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:45.863318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:45.863815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:45.883762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:45.885662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:45.894014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:45.948563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c7011-4819-4d65-9a0f-af4c5d6db022 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:45.955562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 392/795] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:45.956690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.007991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.008656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.008925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.009196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.028127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.029646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:46.049428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:46.050368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:46.051317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:46.056527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 404/796] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:46.057473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.066019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe3b9f-3aac-46d0-897d-352970b46d85 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.067791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.105568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.106181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.106460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.106730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.107378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:46.107675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.116695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.117919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 393/797] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:46.124930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.249717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1635b65-fbd3-47ee-9c54-1dac1a7ee10b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.260684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.279614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c94c04-4d18-43e1-aa56-b13493eed090 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.313794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.314401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.314726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.314993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.315668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:46.315960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.317403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.318067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.318807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.319183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.319545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.330220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 405/798] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:46.330927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.346718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.348524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:46.369065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:46.370062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:46.370988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:46.370988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:46.370988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:46.370988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 394/799] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:46.371222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.400699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.458123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22033c88-8a8c-4d2e-9305-4daf79869fe3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.481344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10f619-830c-4167-9bd3-cb99081e9594 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.504038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.505118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.505594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.506057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.517584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:46.519595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.520240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.520597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.520872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/40dc194ca31947339986b511f4271498/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.522903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=40dc194ca31947339986b511f4271498, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:46.524344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:46.525489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:46.526624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:46.527418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.533981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 406/800] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:46.533981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.544892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 395/801] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:46 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/40dc194ca31947339986b511f4271498/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:46.546232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.590617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.708682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.741113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50583c4d-f48a-44c8-864d-3217487c81c2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.758214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c910b30-ad18-4a76-9143-4a5da328a72e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.793407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.794013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.794280 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.794541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.795174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:46.795470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.796972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.806088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.806702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.807536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.807878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.808877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:46.809185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.810385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 407/802] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:46.810557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.817016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.826956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 396/803] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:46.833410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:46.921499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:46.938430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d961ab5-4a4b-4fbb-afe1-687baaadfec4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.967804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4d9129-e657-4a10-84c6-bd78c9d29af3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:46.974282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:46.975130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:46.975841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:46.976333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:46.983935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:46.985692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-c0df863a-15af-4d88-87e3-83439263510c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:46.997738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:46.998706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:16:47.000189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:47.002355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 408/804] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:47.003711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc7b3-9f97-4e43-97fa-dbb1c1143f92 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.004206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc7b3-9f97-4e43-97fa-dbb1c1143f92 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.004698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4dc7b3-9f97-4e43-97fa-dbb1c1143f92 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.006381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 409/805] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:47.008571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dc7b3-9f97-4e43-97fa-dbb1c1143f92 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.010881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.011603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.011884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.012180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.027552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.029132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.032636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-6253a0ba-a5ce-427e-b03b-f55c2d939e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:47.049466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:47.050473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:47.051404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:47.051404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:47.051404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:47.051404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 397/806] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:47.052149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.057853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.058092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.085393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:47.086728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:47.124380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4dc7b3-9f97-4e43-97fa-dbb1c1143f92 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.159293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.160141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.160622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.161049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.161895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:47.162333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.164002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.167395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 398/807] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:47.168310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.183856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 410/808] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:47.190739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.194833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.288187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e934fc-b63f-42fc-bd7a-24583bed600e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.313830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.334221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.335166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.336459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.338514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.339383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:47.339901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.341507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.364356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 399/809] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:47.370162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.386467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5f69f-2afd-4192-b1ac-40e71f7e02b1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.419953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.421092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.421365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.421634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.479851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6df08f-5432-47ce-b67b-7d1e72da30a1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.490630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.492125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.495262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.516903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.517535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.517839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.518073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.526505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.527972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e88dc9ed-8e0e-415c-826b-2a3bd9cf4e04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:47.539945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:47.540939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:47.541873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 400/810] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:47.542952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bda417-3727-44e8-8e15-70c8bc298c78 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.542952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bda417-3727-44e8-8e15-70c8bc298c78 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.542952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bda417-3727-44e8-8e15-70c8bc298c78 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.548674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 401/811] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:47.548674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bda417-3727-44e8-8e15-70c8bc298c78 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.593515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.604002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 411/812] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:16:47.604002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.661076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bda417-3727-44e8-8e15-70c8bc298c78 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.698942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.699686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.699968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.700244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] PATH_INFO: `/v3/services/network` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.701152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.702593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.709352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6b740-6f92-4176-a6ed-51d635616170 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.711902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-7e59d75b-7495-4b3c-b918-b4d1113a242c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:47.715538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:47.716622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:47.718008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 402/813] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/services/network => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:47.718815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.719463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.719463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.738989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:47.740168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:47.751577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.752174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.752484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.752838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.753462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:47.753763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.755183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.772836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 412/814] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:47.773433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.816257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 403/815] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:47.817458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.868846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:47.889216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e979e3-f625-472b-bce7-4cfc7ab421fe None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.927670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:47.928417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:47.928759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:47.929101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:47.939401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:47.940973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:47.957591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:47.958557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:47.959446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:47.959446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:47.959446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:47.963466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 413/816] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:47.964272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:47.966529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed21ab-986b-461e-a527-8b7cc1981451 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:47.981862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.018101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.018625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.018860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.019074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.019715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.020027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.023351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.033258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 404/817] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.034381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.087580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a8169-cb28-44ec-851f-a9eeffe41a70 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.125515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.126258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.126534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.126802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.137069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.138506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.147141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.151311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07d50a-840a-49a4-b2cd-1f132f09fac0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:48.153514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:48.154426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:48.160601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:48.160601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:48.160601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:48.160601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 414/818] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:48.160601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.187446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.188040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.188326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.188595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.189330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.189650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.191072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.204262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 405/819] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.205137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.227949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.261748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cb990-138a-4790-a8e4-c28f0dc8ed37 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.298263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.298865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.299137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.299399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.300461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.300764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.302113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.312043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 415/820] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.312689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb33892-dd07-4d11-8da8-30f608f6de60 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.312986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.348200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.348775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.349048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.349316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.349882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.351489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:48.360476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:48.361478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:48.362340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 406/821] 10.222.0.33 () {56 vars in 1129 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:48.363133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.392141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.425169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89539e7-d387-4f74-b9ff-0178bf24fbb1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.455066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.461444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.462047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.462324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.462586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.463213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.463529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.464944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.473114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db206d-3ee5-4063-b2cf-62a593f7ed75 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.481575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 416/822] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.481575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.511069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.511787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.512057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.512345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.521992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.524592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:48.538386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:48.539328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 407/823] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:48.540237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.587533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e51a-4367-4470-9009-ded63cb17dcf None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.623610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.624150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.624745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.624745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.625340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.626685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.631782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.652972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 417/824] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:16:48 2022] POST /identity/v3/endpoints => generated 337 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:48.653789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.660740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca59216-642f-4b79-8254-8b19445f00a4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.708636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.709378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.709991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.709991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.710668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.710975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.712467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.723467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 408/825] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.724516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.736219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.807839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44fe2-42f7-4ce2-a81d-324ce44b3db9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.846087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.846722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.846997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.847268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.855679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.857307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.865885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf958f-1356-4674-93f5-6ab28030fce2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:48.875625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:48.876609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:48.878454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 418/826] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:48.879252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.918613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:48.919241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:48.919522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:48.919864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:48.921132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:48.921471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:48.922994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:48.936185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 409/827] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:48.937558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:48.941841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:48.993659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93026cee-7fa4-4e8d-9840-4c857e3d173f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.001565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:49.029240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.029843 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.030652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.030920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.031534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:49.031825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.033214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:49.037796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 419/828] 10.222.0.33 () {56 vars in 1124 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:49.038551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:49.048739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9d22-2d15-4496-9a46-ed34df605b24 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.081076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.081664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.081936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.082204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.082875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:49.083174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.084609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6acbc-e493-4aa0-bb99-ca5ecc19ec93 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:49.096329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 410/829] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:49.102320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c43e-c118-4631-bd01-2d0c5426cc48 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.102601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c43e-c118-4631-bd01-2d0c5426cc48 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.102872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec0c43e-c118-4631-bd01-2d0c5426cc48 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.104073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 411/830] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:49.104831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0c43e-c118-4631-bd01-2d0c5426cc48 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:49.137337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0f58b-51cf-4d80-93c6-fbf35731a5d1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.164144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:49.169822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.170416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.171226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.171497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.172204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:49.172531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.173954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:49.179471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 420/831] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:49 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:49.180208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:49.206018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec0c43e-c118-4631-bd01-2d0c5426cc48 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.228648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:49.247284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.248024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.248396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.248771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.249511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.250970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c27ae-ecbf-4061-8287-9aaf8536f6b2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:49.274848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 412/832] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:16:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:49.276504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.276968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.277238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.298268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:49.299548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:49.336807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8563b9ed-3823-4fde-90b7-81b45ab35d31 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.395938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.397086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.397517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.397880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/2fd208a28c9d41288c294cdf01bf6a31/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.402551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 413/833] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:49.420593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=2fd208a28c9d41288c294cdf01bf6a31, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.422210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:49.439846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 421/834] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:16:49 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/2fd208a28c9d41288c294cdf01bf6a31/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:16:49.674719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:49.726619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:49.780349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dfe9fd-652d-4e19-a286-7c902c7d8711 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.815860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.816523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.816798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.817066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.827987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:49.832780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:49.833697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:49.834548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 414/835] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:49.841475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:49.932900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:49.953792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155e1fb-88c1-420f-9eac-5c3192542566 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:49.989035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:49.989646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:49.989918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:49.990185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:49.990832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:49.991126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:49.992504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.001825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 422/836] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:50.007244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:50.114470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6e64-cdf0-477e-bd73-785938ae8c0f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:50.124423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:50.150673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:50.151642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:50.152125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:50.152596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:50.163089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:50.164764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:50.181067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:50.182069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:50.183014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:50.183014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:50.183014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:50.183014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 415/837] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:50.188468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:50.297196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b73d06-da98-4412-99a0-526a3a397aaa None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:50.332443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:50.333041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:50.333311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:50.333580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:50.334196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:50.334488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:50.335833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.348659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 423/838] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:50.357356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:50.462391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00b754-35ec-4884-adca-d7022b9ef783 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:50.497441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:50.498059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:50.498330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:50.499124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:50.506540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:50.507948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:50.519516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:50.521442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:50.523690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 416/839] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:16:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:50.527267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:50.530945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:50.628494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7b698-a796-4d90-a464-1884eab265bd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:50.662711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:50.663316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:50.663589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:50.663857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:50.664608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:50.664907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:50.666388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.688658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 424/840] 10.222.0.33 () {56 vars in 1135 bytes} [Thu Sep 22 12:16:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:50.690857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:50.714265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:50.794518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7787-5e70-4c37-86ac-2e1bf1b161e9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:50.828836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:50.829443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:50.829874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:50.830191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:50.904562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:50.906036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:50.926877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.009000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 417/841] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:16:51.014240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.073182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.115381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dec8c3-60a0-42fa-bba3-3b9f3644d5d2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:51.151410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:51.152031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.152321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.152586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.162754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:51.164185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:51.179700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:51.181952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:51.184075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:51.184075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:51.184075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:51.184075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 425/842] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:51.189475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.291797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c9d9d-f028-4d61-8681-5099087c2ab6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:51.320694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.332258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:51.332952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.333227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.333494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.334136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:51.334432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:51.335793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:51.349062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 418/843] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:51.357370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.464161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a4926-a3dc-4de9-8b85-ec1f5c2bce54 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:51.478695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.501286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:51.501910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.502222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.502493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.509195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:51.510679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-845a7987-a644-4d08-8e2d-e871c83929a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:51.521953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:51.523994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:51.525510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 426/844] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:51.529086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.540150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cff3c-9f85-4225-92b0-a47c6d802e6c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.540687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cff3c-9f85-4225-92b0-a47c6d802e6c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.540687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601cff3c-9f85-4225-92b0-a47c6d802e6c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.542360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 427/845] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:51.548746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.549025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.549292 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.571557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:51.572912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:51.650198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3d69-595c-4369-927a-e721c34061a7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:51.672188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 428/846] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:51.690996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:51.691637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.691915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.692182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.692933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:51.693229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:51.694704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071206a-5555-4cef-b938-dc0b04ee3b47 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:51.708016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 419/847] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:51.718014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.741008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a963-2518-43a5-be80-4a985f411ee0 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.741290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a963-2518-43a5-be80-4a985f411ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.741566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09a963-2518-43a5-be80-4a985f411ee0 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.743138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 420/848] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:16:51.748471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8796ca-e8b1-4b75-9593-939210cc2bf7 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.748768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8796ca-e8b1-4b75-9593-939210cc2bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.749043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8796ca-e8b1-4b75-9593-939210cc2bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.767263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-cf8796ca-e8b1-4b75-9593-939210cc2bf7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:51.768478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cf8796ca-e8b1-4b75-9593-939210cc2bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:51.807887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.821320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee34c7e-78f3-4e0d-8951-b4497bd611bb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:51.822932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:51.858615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:51.859098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 421/849] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:51.859740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.860187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.860653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.860772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349701af-e94d-43f5-a0ec-018b92ab2423 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.861007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349701af-e94d-43f5-a0ec-018b92ab2423 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.861362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349701af-e94d-43f5-a0ec-018b92ab2423 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.862065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=40dc194ca31947339986b511f4271498) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:51.862539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:51.863080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 422/850] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:51.864486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:51.868156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:51.868462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:51.868733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:51.880338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 429/851] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/role_assignments?user.id=40dc194ca31947339986b511f4271498&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:51.902574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:51.904138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:51.944883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:51.980952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 423/852] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:52.057760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ca159-fd4b-44a8-b3ec-40eb7bba90f2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.094990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.095625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.095901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.096169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.096792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.098155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:52.107264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:52.108261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:52.109526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 430/853] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:52.114477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.130939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.163322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.238589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.244105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3eb06-66f9-4de5-ba86-5491d025a788 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.245715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7141d-293c-4095-be09-41cb9b1c5621 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.280614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.281218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.281494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.281667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.281777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.282285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.282382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:52.282562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.282664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.282832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.283982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.288921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 424/854] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:52.289554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.298668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:52.303957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:52.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 431/855] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:52.306511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.400187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.402766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e9451-16fb-46f9-b59f-0d712b6b73c2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.411087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e659e3cc-1429-47c2-b8dc-25dab8c4789a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.438867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.439650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.440094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.440464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.441095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.442507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.444676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.445277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.445546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.445808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.446414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:52.446703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.448037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.452432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 432/856] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:52.452892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:52.453803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:52.454681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:52.455469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.457110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 425/857] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:52.463221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.508701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.584832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee9750-46c9-4232-b81d-0b0903e75143 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.588948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab32b5-d04a-4c7a-b37e-f8838c5cdf3a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.628222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.629060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.629472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.629837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.630033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.630690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:52.630957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.631166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.631525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.631938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.632748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:52.632953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.633328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.634857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.646813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 426/858] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:52.646813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.647591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 433/859] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:52.648713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.667659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.762446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:52.764641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a95366-2f5a-48a9-b1df-fd822fb20f4d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.775279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248a5c0-988a-4299-9dbd-be7042a94a4b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:52.807394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.808059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.808354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.808661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.811869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:52.812744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:52.813217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:52.813695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:52.814503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:52.814996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.816680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.820739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:52.822183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:52.822396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 427/860] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:52.838181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:52.839195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:52.840085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:52.840085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:52.840085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:52.840085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 434/861] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:52.846287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:52.969901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e29a9-2b30-45ba-8d52-2ec7571ec299 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:53.008404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:53.009017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:53.009297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:53.009570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:53.010208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:53.010561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:53.011936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:53.024823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 428/862] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:53.030933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:53.099719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:53.145809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75593bb8-5a9f-4059-b856-9299da470a0a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:53.180614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:53.181512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:53.181940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:53.182360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:53.190610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:53.192249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:53.204529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:53.205760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:53.207005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:53.209216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 435/863] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:16:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:53.220417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:53.240673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:53.325705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf132a3-dba8-4fe9-a188-d57acd223643 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:53.362904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:53.363512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:53.363792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:53.364063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:53.364814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:53.365116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:53.366538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:53.388476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 429/864] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:16:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:53.390643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:53.495294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa6dc1-1381-4730-b980-b73169242f7c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:53.522132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:53.530339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:53.531202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:53.531548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:53.531968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:53.600606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:53.602123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:53.704161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 436/865] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:16:53.709892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:53.789707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:53.815526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dfe9e8-e59f-4913-b13d-44ec8ac9aeba None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:53.853060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:53.853695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:53.854026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:53.854355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:53.864889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:53.866355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:53.881874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:53.882847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:53.884378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:53.884378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:53.884378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:53.884378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 430/866] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:53.889095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:53.924735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:53.991744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04177b-30a7-402a-9fb4-cf00dff7ab9d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.026099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.026708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.026986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.027258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.027886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:54.028243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.029724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.041662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 437/867] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:54.047643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:54.155339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dacd296-cc50-4188-979d-e3cfd8b99411 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.181929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:54.191730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.192386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.192670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.192945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.199900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.201376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-4f413169-ba4f-4812-8355-23dbe3cf697d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:54.213228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:54.214148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:54.215443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 431/868] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:54.220417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:54.233688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006909a-f8d5-4c1c-bd53-7eaf407682b8 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.233989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006909a-f8d5-4c1c-bd53-7eaf407682b8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.234230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006909a-f8d5-4c1c-bd53-7eaf407682b8 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.235861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 432/869] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:54.241818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.242455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.242662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.263038 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:54.264273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:54.320693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:54.330129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb3c84-b9d6-42df-a6ab-2039ccf2acf2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.348449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 433/870] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:54.370813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.371482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.371761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.372080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.372834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:54.373132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.374589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.387152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 438/871] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:54.392814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:54.494019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562f476-9a37-456f-983b-c9bc002bb8d0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.510832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:54.529116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.530436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.530983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.531274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.532000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=2fd208a28c9d41288c294cdf01bf6a31) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:54.532319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.533776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5aba7-9052-4fd3-ba5d-ef5343dcf295 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.540072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 434/872] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:54 2022] GET /identity/v3/role_assignments?user.id=2fd208a28c9d41288c294cdf01bf6a31&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:54.627216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:54.756681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d3440-e38f-4f59-a4b9-e1a0192141cd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.793365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.794045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.794322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.794592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.795193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.796591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-39f58358-33cc-4a16-a2fb-3d77547bb449 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:54.805811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:54.808132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:54.810145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 439/873] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:16:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:54.812129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24df68-1280-42d4-99f7-884bb2aba5e9 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.812129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24df68-1280-42d4-99f7-884bb2aba5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.812129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24df68-1280-42d4-99f7-884bb2aba5e9 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.812129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 435/874] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:54.814505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.814901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.815051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.815485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24df68-1280-42d4-99f7-884bb2aba5e9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:54.837667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:54.837970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:54.847220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:54.919172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 440/875] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:54.946300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24df68-1280-42d4-99f7-884bb2aba5e9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:54.983217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:54.983818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:54.984096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:54.984393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:54.985014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:54.985311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:54.986720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd92465-8411-4039-89dd-cb5e2aaf93ae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:54.991877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 436/876] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:16:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:55.001384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:55.109510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd168c-8689-4609-bc40-520a68cf107e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:55.132999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7fb1-e264-462c-b6e8-c3d73c9cbafe None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:55.133833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7fb1-e264-462c-b6e8-c3d73c9cbafe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:55.134324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd7fb1-e264-462c-b6e8-c3d73c9cbafe None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:55.136721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 437/877] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:16:55.142387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:55.142580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:55.142836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:55.151376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:55.152251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:55.152750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:55.153228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:55.154045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:55.154598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:55.156155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:55.162093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:55.162369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 441/878] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:16:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:55.168338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:55.194439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:55.209202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:55.237066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:55.267037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 438/879] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:55.318915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f5a77-f619-4fe7-9e8a-e1d1147009c7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:55.358820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:55.359727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:55.360004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:55.360276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:55.360956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:55.362321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7012291-eb1c-4d9c-8dd1-bf3b30878d96 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:55.379714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 442/880] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:16:55 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:16:55.534814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:55.641030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386f854-d206-440b-bafb-d6a5f621663b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:55.682815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:55.683422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:55.683697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:55.683968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:55.684647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:55.686111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9932a-2b04-496c-907b-c82c514772a6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:55.704117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 439/881] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:16:55 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:16:56.906109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86919f-36da-42ee-93ef-75ab3f58e170 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:56.907069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86919f-36da-42ee-93ef-75ab3f58e170 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:56.907538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86919f-36da-42ee-93ef-75ab3f58e170 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:56.909390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 443/882] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:56.914836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c480d5-aca8-46a8-a9df-0e91c8cb4065 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:56.915632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c480d5-aca8-46a8-a9df-0e91c8cb4065 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:56.916097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c480d5-aca8-46a8-a9df-0e91c8cb4065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:56.934041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-98c480d5-aca8-46a8-a9df-0e91c8cb4065 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:56.935487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-98c480d5-aca8-46a8-a9df-0e91c8cb4065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:56.960531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:57.011462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 440/883] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:57.279628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86919f-36da-42ee-93ef-75ab3f58e170 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:57.349042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:57.387610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86919f-36da-42ee-93ef-75ab3f58e170 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:57.417603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1d7e-5d41-48aa-a2c1-e8a77e133092 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.417895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1d7e-5d41-48aa-a2c1-e8a77e133092 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.418175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf1d7e-5d41-48aa-a2c1-e8a77e133092 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.419726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 441/884] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:57.422713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:57.423335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.423606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.423868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.426720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.427224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.427712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.434229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:16:57.440023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:57.441035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:57.445160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 444/885] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:16:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:57.447449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:57.456155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:57.458231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:57.552570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 442/886] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:57.564230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7048c-6302-4b5a-aa71-aec94ec58843 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:57.591170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:57.601946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:57.602862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.603351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.603823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.604703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:57.605216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:57.606768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869caa6e-a46a-4309-841b-9e9d8ecae6a2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:57.615978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 445/887] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:16:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:57.621508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:57.657000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d156944-345a-4d35-83a3-7445749b5cbf None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.657284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d156944-345a-4d35-83a3-7445749b5cbf None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.657741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d156944-345a-4d35-83a3-7445749b5cbf None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.659213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 446/888] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:16:57.664908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.665417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.665888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.686731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:57.688177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:57.715378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:57.733823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec11e0-ba72-41b2-9156-259e529fd9dd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:57.769689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:57.770378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 447/889] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:16:57.777008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.777469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.777911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.789357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:57.791023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:57.805918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:57.806856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:57.807739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:57.807739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:57.807739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:57.813872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 443/890] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:57.815356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:57.853182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:57.928660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905423-63ca-437e-8948-8c67ec1e1a46 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:57.947293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:57.967568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:57.968172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.968477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.968745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.969366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:57.969658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:57.971008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7b28d-5e59-4cff-94d7-5d90bb7a8074 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:57.977679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d256bc7-672c-4316-aed3-26960168f514 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:57.983173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 448/891] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:57.984537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbaac-2ab0-45c8-a16c-bfd0d870c24b None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:57.984986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbaac-2ab0-45c8-a16c-bfd0d870c24b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:57.985407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9bbaac-2ab0-45c8-a16c-bfd0d870c24b None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:57.987560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 449/892] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:16:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:16:57.989376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bbaac-2ab0-45c8-a16c-bfd0d870c24b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.014929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.015647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.015923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.016197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.016862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.018354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba0aa9-a85c-4abc-a5dd-8ca0444e31f5 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.028100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.043447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 444/893] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:16:57 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:16:58.043447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.043758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.043850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.064355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:16:58.065497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:16:58.110493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9bbaac-2ab0-45c8-a16c-bfd0d870c24b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.142252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 445/894] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:16:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:16:58.143456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.163168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.165015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.165585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.166103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.178182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.180020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:16:58.193818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:58.194836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:16:58.195801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:58.203817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 450/895] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:16:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:16:58.205497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.255456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.274997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88145d7c-b2a5-4961-830a-8e1907e1b2f0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.310052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.310941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.311302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.311863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.313746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.315076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:58.326082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:58.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:58.328416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 446/896] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:58.329318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.333073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.348554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a094a1-77c5-4c55-ac56-882e4fb1cf93 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.395738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.396787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.397541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.398274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.399182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:58.399750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.401969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.427681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 451/897] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:58.429688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.453006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a478bf4-046b-49b1-a7c3-cd170f518181 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.489787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.490391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.490671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.490944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.491593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:58.491903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.493288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.497942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 447/898] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:58.498725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.539087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.549796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae26f344-7b60-492e-8518-b8803f859f72 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.558218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.587504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.588252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.588543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.588810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.589678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.590736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:58.600043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:58.601266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:58.602823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 452/899] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:58.603662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.615513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83760486-0944-42b1-8bec-ab7eb1b360c2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.652276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.653191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.653473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.653746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.703128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5b87c-f4fe-4e0c-9333-6e3b66b04b9c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.727396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.728879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.738235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.739105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.739383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.739648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.740257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:58.740585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.741929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.746735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 453/900] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:58.746735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.772378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.798357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:58.852417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 448/901] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:16:58.852417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.866452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9a4a6-b7a3-4496-ab8f-c1d5c2e0f878 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:58.904145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:58.904954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:58.905235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:58.905504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:58.906123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:58.906423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:58.907781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:58.912447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 454/902] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:58.913211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:58.970233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6141e7-6c35-478b-828c-9e844e337864 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.005228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.006459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.007064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.007557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.007557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.008274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] RBAC: Adding query filter params (interface=public, service_id=6b91342e43184b31bb33d760e9f5ca0a, region_id=RegionOne) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.008607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.009953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.020205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 449/903] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/endpoints?service_id=6b91342e43184b31bb33d760e9f5ca0a&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:59.020205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.028051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cb7fa-a0e9-492a-916c-c48f3b79a818 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.041348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.062368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.063000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.063275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.063549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.078781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.080235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:16:59.095015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:59.095991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:16:59.096930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:16:59.096930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:16:59.096930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:16:59.096930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 455/904] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:16:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:59.098215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.136239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae3c36-49a4-43d9-9dac-a9cf29a41d6c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.171861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.172561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.172834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.173099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.173712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.174013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.175365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.180728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 450/905] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:59.181494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.212286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6b489-c8aa-4724-b8ff-0302f6e0c3f5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.250254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.250864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.251141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.251409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.252004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.253391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.255550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.269536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 456/906] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:16:59.270837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.280982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.299956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b375c1f3-3ba2-4055-876c-a6bac4204382 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.354420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.356097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.356738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.357337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.358993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.359500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.361892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.381329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 451/907] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:59.387379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.419301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda9962-9129-4952-9032-119a1cc21ad7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.456738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.457365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.457628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.457919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.458749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] RBAC: Adding query filter params (interface=public, service_id=7bed008afcd64183af9e0455130c3f5c, region_id=RegionOne) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.459116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.461226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.466487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 457/908] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/endpoints?service_id=7bed008afcd64183af9e0455130c3f5c&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:16:59.472815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.517885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a7300-a22d-47c3-9785-bc208ef47625 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.553366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.557796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.558425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.558698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.558957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.566275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.567703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:16:59.579448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:16:59.580421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:16:59.581332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 452/909] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:16:59.586471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.590957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.608208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da853147-c29b-4c61-8230-61d18f139d35 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.653587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.654110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.654924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.655199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.656348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.658031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.673003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 458/910] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:16:59.715816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767af66-134b-4e7e-bf7c-c67b9c9ae053 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.757857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.758472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.758747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.759017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.759709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.760013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.761519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fcd85b-1749-4fa3-8420-233291c28dce None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.775577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 453/911] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:16:59.782552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:16:59.913218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c71c06-7a30-4d60-8cbd-4015ef366b23 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:16:59.940356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:16:59.952667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:16:59.953415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:16:59.953692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:16:59.956316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:16:59.956316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] RBAC: Adding query filter params (role.id=036a2147287c4f9eb1fc966fb2da43df, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=2fd208a28c9d41288c294cdf01bf6a31) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:16:59.956316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:16:59.956578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7ce69-a500-43b9-80b5-4bdb716b3b40 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:16:59.963430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 459/912] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:16:59 2022] GET /identity/v3/role_assignments?user.id=2fd208a28c9d41288c294cdf01bf6a31&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=036a2147287c4f9eb1fc966fb2da43df => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:00.254515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88753ebf-3bc2-4cca-a4d6-5fd670d468f1 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:00.255024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88753ebf-3bc2-4cca-a4d6-5fd670d468f1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:00.255024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88753ebf-3bc2-4cca-a4d6-5fd670d468f1 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:00.256618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 454/913] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:00.262613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:00.263120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:00.263120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:00.282193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:00.283369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:00.352845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 460/914] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:00.618495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88753ebf-3bc2-4cca-a4d6-5fd670d468f1 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:00.687935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:00.728076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88753ebf-3bc2-4cca-a4d6-5fd670d468f1 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:00.762866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:00.763486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:00.763834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:00.764110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:00.764708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:00.766045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:00.774786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:00.775764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:00.776643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 455/915] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:00.782009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:00.887830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fca300-a99d-4901-bf78-8f34308d38ad None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:00.912735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:00.926338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:00.926943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:00.927213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:00.927476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:00.928090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:00.928413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:00.929769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:00.935066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 461/916] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:00.941017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:01.049410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb3cc7-0f54-482d-8719-8b512317a9cc None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:01.080085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:01.084401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:01.085236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.085735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.086610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.087562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:01.088083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:01.089647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:01.095659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 456/917] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:01.101604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:01.202060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15bd7e-6c22-4bc0-b130-caab7d7e3f96 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:01.237022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:01.237620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.237891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.238155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.238837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] RBAC: Adding query filter params (interface=public, service_id=e463915baa5b4ad4b0964e97ff4ba70f, region_id=RegionOne) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:01.239138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:01.240507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584ac4b-d0d2-4324-9480-a05e063714ae None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:01.245823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 462/918] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:17:01 2022] GET /identity/v3/endpoints?service_id=e463915baa5b4ad4b0964e97ff4ba70f&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:01.251196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:01.358089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8441f22-2cf1-400e-9a49-9f6c7b84f6f4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:01.393870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:01.394799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.395300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.395792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.396612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:01.398162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9740edec-f725-4e5b-98a0-4471a7c4160b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:01.405312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 457/919] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:17:01 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:17:01.564897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0176c6-e6de-4a30-a0f4-a989da66f472 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.565451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0176c6-e6de-4a30-a0f4-a989da66f472 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.565451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0176c6-e6de-4a30-a0f4-a989da66f472 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.567046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 463/920] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:01.572692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c965-7a90-4658-8d56-b337568a94bf None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.572997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c965-7a90-4658-8d56-b337568a94bf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.573244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60c965-7a90-4658-8d56-b337568a94bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.593484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ac60c965-7a90-4658-8d56-b337568a94bf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:01.594679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ac60c965-7a90-4658-8d56-b337568a94bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:01.650157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:01.668076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 458/921] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:01.943419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0176c6-e6de-4a30-a0f4-a989da66f472 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:01.991230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407fe7-6bf6-4971-a3e0-2aeef024622e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:01.991537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407fe7-6bf6-4971-a3e0-2aeef024622e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:01.991783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407fe7-6bf6-4971-a3e0-2aeef024622e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:01.993409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 459/922] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:01.999458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:01.999780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30e75-2164-4afe-8779-30e10208f768 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.000061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30e75-2164-4afe-8779-30e10208f768 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.000356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30e75-2164-4afe-8779-30e10208f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.024706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c6b30e75-2164-4afe-8779-30e10208f768 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:02.026388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c6b30e75-2164-4afe-8779-30e10208f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:02.055771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0176c6-e6de-4a30-a0f4-a989da66f472 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.093352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.093982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.094255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.094517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.095191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.096588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:02.106353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:02.107612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:02.108690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 464/923] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:17:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:02.115010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 460/924] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:02.115929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.229996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331e5b5-cd55-4de8-876e-d5ca14f1b1b1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.244190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:02.251612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36ed7a-a606-44a8-a1bd-33205cd2aeaf None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.251901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36ed7a-a606-44a8-a1bd-33205cd2aeaf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.252181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36ed7a-a606-44a8-a1bd-33205cd2aeaf None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.253775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 461/925] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:02.258918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.259201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.259479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.266980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.267829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.268275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.268756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.269539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:02.270007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.271533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.276898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 465/926] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:02.278681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:02.279965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:02.284408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.364627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 462/927] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:02.394021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f88af3-c559-4f7a-bccf-d67bc7bc5801 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.417369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.428973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.429651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.429992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.430327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.431044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:02.431406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.433251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.440338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 466/928] 10.222.0.33 () {56 vars in 1114 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:02.445664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.485398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:02.499317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:02.559818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbf3bc7-f528-4a6a-91d0-f0b11042c325 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.570471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f0cb8-d197-471b-8550-3ba8008e51c6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.604861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.605080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.605534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.605810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.606078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.606454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.606694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.607961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.608424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.608636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.609415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.610918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:02.618753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:02.619707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 12:17:02.620640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:02.624509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 463/929] 10.222.0.33 () {56 vars in 1132 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:02.626407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.642122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 467/930] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:02.643101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.757202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bfc346-35e3-4ed3-98c8-ed6f9d15e42d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.760983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:02.773546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5502265f-4657-4a4a-82c8-e567a5466e53 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.773810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:02.803677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.804527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.804898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.805257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.806140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:02.806537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:02.808139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:02.816392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 464/931] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:02.819622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.820233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:02.820929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:02.821205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:02.821470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:02.834389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:02.839543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:02.840937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:02.842739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 468/932] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:02.847765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:02.961433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d622a-9000-462f-9d1a-1da0da544323 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:02.967835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4de1993-94b5-4d3d-959c-38ed5a3260e3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.007071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.007733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.007954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.008226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.008901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:03.009203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.010572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.013154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.014804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.015285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.015769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.016349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 465/933] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:03.016858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:03.017393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.018959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.021894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.030592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 469/934] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:03.036364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.056481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:03.068832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:03.155477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457cf77-0a71-4602-94ca-482a02d482ef None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.158886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd1ec5-b98b-487d-ac21-874f47a54ec0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.190886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.191905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.192430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.192981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.196182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.196942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.197264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.197542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.198258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.199618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.207216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.209044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:03.227017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:03.228166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:03.229495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:03.229495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:03.229495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:03.236319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 466/936] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:17:03.236515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 470/936] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:03.237469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.349492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:03.363722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04950fd9-5425-4d14-938a-86c5d6ae4494 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.402197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.402855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.403066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.403338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.403984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:03.404345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.405788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.418948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 467/937] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:03.424836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.501702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:03.546122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7de10e-7152-4892-9b91-6857853384d2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.582830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.583582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.583770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.584037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.592484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.593909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ed1654ef-1648-4095-95ff-8190cb4fddc4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:03.605505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:03.606433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:03.607940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 471/938] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:03.612909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.650299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d837-a58a-41e7-a99a-5685b9055ea3 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.650617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d837-a58a-41e7-a99a-5685b9055ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.650841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1183d837-a58a-41e7-a99a-5685b9055ea3 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.652510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 472/939] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:03.661971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.662251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.662528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.682859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:03.684069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:03.731820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcc021-d771-4be3-9fd4-49a6c28ddc0c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.764554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 473/940] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:03.774965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.775571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.775850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.776120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.776867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:03.777169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.778584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.798068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 468/941] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:03.803773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:03.877450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:03.921923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed485e23-76c9-4f3e-868a-c36712c1a95d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:03.957772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:03.958677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:03.959174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:03.959638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:03.971032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:03.972730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:03.988099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:03.989109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:03.990003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:03.990003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:03.990003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:03.991744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 474/942] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:03.997457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.013330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.048220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.115012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677a4e-7b2d-4a4a-933e-38af08a1d463 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.165475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.166272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.166551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.167075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.168134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:04.168480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.170504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.174499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7288c9db-6521-468f-8f8c-da5f8646952f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.187764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 469/943] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:04.190744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.192919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.225325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.226057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.226336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.226609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.227263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.229054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:04.243094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:04.244136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:04.245045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:04.250282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 475/944] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:04.252373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.285513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.332008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc795c75-5ce8-4ee5-9391-b2a564a043db None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.372680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.373858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.374113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.374387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.381165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.382601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.385490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903d5a1-23eb-44b9-a76c-e3d94734b218 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:04.394821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:04.395739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:04.396695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:04.400671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 470/945] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:04.401991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.425967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.426957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.427471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.427940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.428788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:04.429353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.431031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.437607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 476/946] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:04.443232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.506204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.529774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227c20f-6a48-49f0-b350-c2fdccabb2f6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.560075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.570292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.570922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.571571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.571850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.572861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:04.573174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.574602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.593912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 471/947] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:17:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:04.595143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.598052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c575ea0e-962f-45a6-9821-58e85fdbbf55 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.636562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280359c7-215c-454f-b785-184ca5763bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.637167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280359c7-215c-454f-b785-184ca5763bad None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.637443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280359c7-215c-454f-b785-184ca5763bad None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.638940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280359c7-215c-454f-b785-184ca5763bad None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.639594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280359c7-215c-454f-b785-184ca5763bad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:04.639899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280359c7-215c-454f-b785-184ca5763bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.641382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280359c7-215c-454f-b785-184ca5763bad None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.651154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 477/948] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:04.652411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280359c7-215c-454f-b785-184ca5763bad None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:04.726450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30234fc-1291-49ce-975c-20d526a98b30 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.762149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.762957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.763240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280359c7-215c-454f-b785-184ca5763bad None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:04.763477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.763849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/2fd208a28c9d41288c294cdf01bf6a31/roles/036a2147287c4f9eb1fc966fb2da43df` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.765443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:04.766680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=2fd208a28c9d41288c294cdf01bf6a31, role_id=036a2147287c4f9eb1fc966fb2da43df)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.768218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8782b1c2-ea77-433a-acc0-eb017d632ca0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.785976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 472/949] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:17:04 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/2fd208a28c9d41288c294cdf01bf6a31/roles/036a2147287c4f9eb1fc966fb2da43df => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:17:04.819324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:04.820406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.820837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.821258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.822080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:04.823654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353f44f-1406-4d1e-a12a-4555dac5c0a0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:04.848834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 478/950] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:04.983508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9476f-36fe-4fd9-b9fb-605a24b388e0 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.983687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9476f-36fe-4fd9-b9fb-605a24b388e0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.983960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9476f-36fe-4fd9-b9fb-605a24b388e0 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:04.985632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 473/951] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:04.991403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9f7b-a28f-43ca-be24-3af6cf44243d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:04.991713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9f7b-a28f-43ca-be24-3af6cf44243d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:04.991954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8e9f7b-a28f-43ca-be24-3af6cf44243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.012712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8c8e9f7b-a28f-43ca-be24-3af6cf44243d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:05.014001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c8e9f7b-a28f-43ca-be24-3af6cf44243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:05.095353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 479/952] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:05.373024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9476f-36fe-4fd9-b9fb-605a24b388e0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:05.428146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:05.484339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9476f-36fe-4fd9-b9fb-605a24b388e0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:05.521107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:05.521985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:05.522444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:05.522875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.523710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:05.525407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:05.528317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING keystone.server.flask.common [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 12:17:05.528492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427402f-e058-4dc3-aa32-bd8554ee244c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:05.528779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427402f-e058-4dc3-aa32-bd8554ee244c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:05.529079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427402f-e058-4dc3-aa32-bd8554ee244c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.530697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 480/953] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:05.536575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:05.536856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:05.537124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.552502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.sql.core [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 12:17:05.552502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 12:17:05.552502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [parameters: {'id': '7db0adeca7b04d4b85546cbc71768528', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 12:17:05.552502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81376) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 12:17:05.562104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:05.563969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7db0adeca7b04d4b85546cbc71768528.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7db0adeca7b04d4b85546cbc71768528. Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:17:05.578651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application result.read() Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 12:17:05.579548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application yield resource Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise self.value Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application session.commit() Sep 22 12:17:05.580545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self.session.flush() Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application compat.raise_( Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 12:17:05.581446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 12:17:05.582411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application result.read() Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:17:05.583223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [parameters: {'id': '7db0adeca7b04d4b85546cbc71768528', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:05.584110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:05.584932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7db0adeca7b04d4b85546cbc71768528. Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 474/954] 10.222.0.33 () {60 vars in 1140 bytes} [Thu Sep 22 12:17:05 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:05.587983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:05.666828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 481/955] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:05.702453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a9c2a-4400-4430-a927-bf55ebde754e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:05.705461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:05.745744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:05.746403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:05.746685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:05.746956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.754944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:05.756632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:05.768503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:05.769375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:05.770898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 475/956] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:05.777587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:05.792837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:05.895047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688a87-1741-4cba-adcc-780ac1af2707 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:05.930664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:05.931200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:05.931476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:05.931745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:05.932455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:05.932709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:05.934126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:05.952757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 482/957] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:05.958787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:06.074810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e50146-cc6e-4c42-bf74-201c2a34221e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:06.112998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:06.113642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:06.113921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:06.114196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:06.114782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:06.116166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:06.126346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:06.127659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:06.128714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 476/958] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:06.133671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:06.222787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:06.250162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bffbaf-54d0-420f-94b0-915d47d54da0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:06.285986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:06.286597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:06.286873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:06.287142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:06.287767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:06.288065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:06.289509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195b211-82b9-4403-982f-05180599ca69 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:06.295393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 483/959] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:06.301023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:06.331840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:06.413070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c6166-f0cd-41b4-a87c-50096b63e3ba None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:06.447807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:06.448446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:06.448724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:06.448994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:06.449609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:06.449908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:06.451352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744e31d-331a-4f55-ad8e-840b4b8c2866 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:06.459022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 477/960] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:07.106854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555b7a-7055-432e-bb65-614216afae26 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.107424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555b7a-7055-432e-bb65-614216afae26 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.107424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10555b7a-7055-432e-bb65-614216afae26 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.109040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 484/961] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:07.114538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.114926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.115083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.134480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:07.135918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:07.171123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872623-83ea-4422-b313-9ecec667f260 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.171391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872623-83ea-4422-b313-9ecec667f260 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.171667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b872623-83ea-4422-b313-9ecec667f260 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.173295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 485/962] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:07.178441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.178719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.178988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.198119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:07.199488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:07.208437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 478/963] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:07.269958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 486/964] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:07.481546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.505867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:07.538452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.588207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d08b1f-c8a9-440b-b030-1d6657023efc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:07.610712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:07.628333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:07.629052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.629401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.629738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.641510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:07.647160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:07.648107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:07.653160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 479/965] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:07.655397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.658110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca9fef-fe7a-4786-a916-d0763eabcc97 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:07.701760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:07.702393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.702669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.702937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.704318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:07.705694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:07.715979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:07.717101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:07.717977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:07.720374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 487/966] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:07.723125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.764102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:07.784554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176698c-2baa-49e1-bf51-aff917d1de19 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:07.825629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:07.826404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.826688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.826969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.828186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:07.828554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:07.829934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:07.833741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3bb61-735d-4e91-8d83-fe28b226d5d1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:07.847898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 480/967] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:07.850148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.873078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:07.883512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:07.884567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:07.885201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:07.885790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:07.887199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:07.887604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:07.889702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:07.900377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 488/968] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:07.902375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:07.974386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071c8b3-7f11-44f3-a1f3-6199f54eb16e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.013862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04134964-1255-4848-b756-a678207ad62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.014567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134964-1255-4848-b756-a678207ad62f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.014779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134964-1255-4848-b756-a678207ad62f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.015047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04134964-1255-4848-b756-a678207ad62f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.028131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04134964-1255-4848-b756-a678207ad62f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.030842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04134964-1255-4848-b756-a678207ad62f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.032618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808243-6324-4494-af03-f67cd8721ce8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.033783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-04134964-1255-4848-b756-a678207ad62f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:08.049138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:08.050161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:08.051077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:08.051077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:08.051077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:08.052375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 481/969] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:08.056840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04134964-1255-4848-b756-a678207ad62f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.082155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.082765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.083035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.083308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.084480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:08.084788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.086460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.093715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 489/970] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:08.099529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.183498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.193195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04134964-1255-4848-b756-a678207ad62f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.233487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.234172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.234549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.234918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.235635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:08.236027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.238253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.240689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68811ef-97c1-4158-83ac-5a6a77671593 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.256374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 482/971] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:08.258254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.281653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.282185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.282458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.282729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.283426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] RBAC: Adding query filter params (interface=admin, service_id=e463915baa5b4ad4b0964e97ff4ba70f, region_id=RegionOne) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:08.283730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.285119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2c628-d34f-4725-abe1-d2d21b1a0f5c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.291298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 490/972] 10.222.0.33 () {56 vars in 1249 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/endpoints?service_id=e463915baa5b4ad4b0964e97ff4ba70f&interface=admin®ion_id=RegionOne => generated 193 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:08.292495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1e44-5ab1-4b9e-9b8b-32d0c751a94c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.292781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1e44-5ab1-4b9e-9b8b-32d0c751a94c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.293055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f1e44-5ab1-4b9e-9b8b-32d0c751a94c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.296424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 491/973] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:08.297210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f1e44-5ab1-4b9e-9b8b-32d0c751a94c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.323137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.371074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a7f20-52b5-404b-9140-16441ce05b76 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.403613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f1e44-5ab1-4b9e-9b8b-32d0c751a94c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.409716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.410345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.410623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.411063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.419136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.420948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.426397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-017b18f9-c364-46fa-9d6a-7af437534ec8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:08.433585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:08.434572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:08.435558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 483/974] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:08.436640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.436640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.436640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.443559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.444167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.444471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.444757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.445341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.446700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.453378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 492/975] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:17:08.454153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.465735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:08.467869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:08.552426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 484/976] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:08.601803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6e123-820f-448f-9f46-3e39aa0b9e88 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.644156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.645144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.645623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.646075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.646982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:08.647465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.649131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9f83b5-50a9-4fbd-9baa-832707cb4727 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.668346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 493/977] 10.222.0.33 () {56 vars in 1149 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:08.675114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.724620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.731052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e1e-8271-4cb7-97e7-a6c95da8c7bf None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.731333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e1e-8271-4cb7-97e7-a6c95da8c7bf None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.731602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76e1e-8271-4cb7-97e7-a6c95da8c7bf None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.733265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 494/978] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:08.740500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.741040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.741515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.763262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:08.765334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:08.791476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8860-1097-47de-9bc5-12da31127e35 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.809839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.832441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.833053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.833333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:08.833609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:08.844133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 495/979] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:08.845546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:08.921193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:08.922826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:08.961547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647c94-be25-43cc-877b-fa809d2b642f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:08.979608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:08.998502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:08.999502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:08.999980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.000494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.001326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.002951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.006103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING keystone.server.flask.common [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 12:17:09.020139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.sql.core [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 12:17:09.020139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 12:17:09.020139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [parameters: {'user_id': 'cc4fc0db9e4b4474942bdfb829e4ffef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 12:17:09.020139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81377) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result.read() Sep 22 12:17:09.028094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:17:09.029055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application yield resource Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application next(self.gen) Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise self.value Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application session.commit() Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 12:17:09.030038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.session.flush() Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application compat.raise_( Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 12:17:09.030945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 12:17:09.031838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application conn.query(q) Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application result.read() Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 12:17:09.032832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cc4fc0db9e4b4474942bdfb829e4ffef', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:09.033682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 12:17:09.034588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 12:17:09.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.040045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 496/980] 10.222.0.33 () {60 vars in 1131 bytes} [Thu Sep 22 12:17:08 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:09.044870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 485/981] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:09.052370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.052505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.091494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:09.154968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd2e43-8893-4627-b40e-4f7a49aa52e6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.160753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa60b4-864f-4303-9a41-32d57e613924 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.192077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.192737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.193012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.193285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.197599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.198215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.198488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.198755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.200821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.202448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.212553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.214002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:09.221470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:09.222741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 12:17:09.223868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:09.225882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:09.227613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 497/982] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:09.228452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:09.235904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:09.236962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:09.237857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:09.237857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:09.237857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:09.241658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 486/983] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:09.242593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.330523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:09.334751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b8488c-5928-40cf-8945-65a389992ed3 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.349662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468e1ae-6146-458b-a6d0-6c4fe1f86699 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.372163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.372830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.373101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.373395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.374263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.375619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.385327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.385942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.386209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.386480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.387167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:09.387478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.388957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.393921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 498/984] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:17:09 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:09.397423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.440803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 487/985] 10.222.0.33 () {56 vars in 1101 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:09.535506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d83c9-259d-43c5-be6b-fde5a38f7b55 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.577489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.578555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.579029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.579496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.580391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:09.580892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.582490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.601251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 499/986] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:09.603593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.668197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:09.716900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ec994c-4d32-41bd-a89a-a279217ae272 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.753174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.753802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.754077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.754349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.761519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.762967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:09.775171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:09.776280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:09.777403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 488/987] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:09.783185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:09.801208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:09.897242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe92b27-16a4-4099-aa65-af45558d9a40 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:09.932027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:09.932719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:09.932997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:09.933268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:09.933992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:09.934299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:09.935719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37edd0-0b88-4757-b414-60cf5f5b540e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:09.954056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 500/988] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:09.955615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:10.056163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c23f0-a3f8-431b-97a2-8cbbf9fc1236 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:10.071928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:10.091469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11934a28-2c34-4b02-b561-31580079387a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:10.092146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934a28-2c34-4b02-b561-31580079387a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:10.092520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934a28-2c34-4b02-b561-31580079387a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:10.092850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11934a28-2c34-4b02-b561-31580079387a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:10.093619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11934a28-2c34-4b02-b561-31580079387a None admin] RBAC: Adding query filter params (role.id=036a2147287c4f9eb1fc966fb2da43df, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=2fd208a28c9d41288c294cdf01bf6a31) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:10.093986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11934a28-2c34-4b02-b561-31580079387a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:10.095477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11934a28-2c34-4b02-b561-31580079387a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:10.101848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 489/989] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:09 2022] GET /identity/v3/role_assignments?user.id=2fd208a28c9d41288c294cdf01bf6a31&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=036a2147287c4f9eb1fc966fb2da43df => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:10.711521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8304-be6d-4170-bdd9-af2f537e4e54 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:10.712065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8304-be6d-4170-bdd9-af2f537e4e54 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:10.712065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8304-be6d-4170-bdd9-af2f537e4e54 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:10.713586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 501/990] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:10.718658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:10.719072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:10.719231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:10.737717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:10.738900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:10.804834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 490/991] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:11.068784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe8304-be6d-4170-bdd9-af2f537e4e54 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:11.145832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:11.172599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe8304-be6d-4170-bdd9-af2f537e4e54 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:11.206017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:11.206680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.206958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.207223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.207838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:11.209212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:11.218606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:11.219596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 12:17:11.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:11.224288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 502/992] 10.222.0.33 () {56 vars in 1126 bytes} [Thu Sep 22 12:17:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:11.226279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:11.322911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5416e2-d25d-49d5-9f10-cd3d320e2f5d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:11.333941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:11.360055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:11.360692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.360969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.361238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.361864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:11.362163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:11.363521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:11.369069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 491/993] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:11.374708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:11.476875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85268dcf-4410-4524-ab31-d7e02431d128 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:11.513796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:11.514418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.514688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.514955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.515586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:11.515887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:11.517328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa0f96-9e8d-4db3-a005-ba03a707b94f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:11.527870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 503/994] 10.222.0.33 () {56 vars in 1122 bytes} [Thu Sep 22 12:17:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:11.529309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:11.613454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4b53-9ce9-4e85-8b5b-7a5341e70494 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.613454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4b53-9ce9-4e85-8b5b-7a5341e70494 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.613454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa4b53-9ce9-4e85-8b5b-7a5341e70494 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.614548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 504/995] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:11.619196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.619477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.619749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.626573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:11.641139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:11.641284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45a17c-b57f-411f-b9e5-e98beb95a1c3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:11.642709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:11.675675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:11.676268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:11.676575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:11.676861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:11.677498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:11.678875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:11.696338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 492/996] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:11.737121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 505/997] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:12.010559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.045987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:12.113063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db2ef9-5491-44a4-b00b-0cc6a0a6d1eb None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.146453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.146984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.147262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.147530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.148097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.149474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:12.158117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:12.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:12.159981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:12.164101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 493/998] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:12.165147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.271971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577d4ff-4404-4d0e-89b8-c29d90e1ef30 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.298499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:12.307516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.308122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.308428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.308698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.309316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:12.309620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.311024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207b58f-e0e1-4530-8c50-5232b504936e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.315887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 506/999] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:12.321305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.326571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54779-cd41-4636-bc10-7e089a221173 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.326851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54779-cd41-4636-bc10-7e089a221173 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.327121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d54779-cd41-4636-bc10-7e089a221173 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.328608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 507/1000] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:12.333171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.333453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.333730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.352376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:12.353576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:12.426647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 508/1001] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:12.429450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:12.433579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b897e5af-1961-48d8-9323-1b02fbd6ea09 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.469229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.469831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.470103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.470368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.470978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:12.471272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.472645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.483430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 494/1002] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:12.484359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.584001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f28ff9-1cdb-4b5c-ab35-41dc1ea84198 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.610008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:12.623002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.623615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.623890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.624161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.624883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] RBAC: Adding query filter params (interface=public, service_id=1de0d685b131458f9870edf36886c245, region_id=RegionOne) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:12.625189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.626553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.636380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 509/1003] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/endpoints?service_id=1de0d685b131458f9870edf36886c245&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:12.637856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.697478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.741705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72373f-45d4-4514-9aa5-106993820bdd None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.778346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.778966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.779240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.779503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.780197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.781682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.799294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c07bc-7472-4ad1-8fe4-7584a018702b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:12.800701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 495/1004] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:17:12.837008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:12.837652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:12.837927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:12.838197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] PATH_INFO: `/v3/services/image` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:12.838780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:12.840185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:12.850225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:12.851332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:12.852186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:12.856605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 510/1005] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:12.858403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:12.890436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:12.973287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259c98-0344-4cf1-91ee-4e7298917abb None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:13.010429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:13.011122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:13.011721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:13.011721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:13.012257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] RBAC: Adding query filter params (name=image) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:13.012580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:13.013951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2af84-5f04-4e75-88b7-3ba3d11823ab None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:13.019210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 496/1006] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:13.025095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:13.076914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:13.128266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdca99c-5492-4059-bcb6-8cc6b5fef824 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:13.161290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:13.161891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:13.162166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:13.162432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:13.163042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] RBAC: Adding query filter params (type=image) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:13.163337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:13.164722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fe2073-7e91-4f21-a70e-07c48c0a1178 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:13.169968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 511/1007] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:13.909483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227fa42-3a88-41a2-bbca-7f2e96a38e0b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:13.909791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227fa42-3a88-41a2-bbca-7f2e96a38e0b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:13.910040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227fa42-3a88-41a2-bbca-7f2e96a38e0b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:13.911530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 497/1008] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:13.916564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:13.917023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:13.917101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:13.935499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:13.936717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:14.007585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 512/1009] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:14.288680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227fa42-3a88-41a2-bbca-7f2e96a38e0b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:14.380395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:14.391937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227fa42-3a88-41a2-bbca-7f2e96a38e0b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:14.429201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:14.429830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:14.430109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:14.430378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:14.431492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:14.433010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d2656-7edf-4926-8ff4-6bc704c7a4e1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:14.434825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 498/1010] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:14.440890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:14.493361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:14.548864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b602947-71a1-4fb2-adbb-7796a101788e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:14.581748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:14.582355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:14.582630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:14.582895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:14.583534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:14.584983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10643a4-cb9d-4d08-b9d2-4332f075a89d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:14.612248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 513/1011] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:17:14 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:14.965945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ef4-5c93-4ad9-a577-c834378a50c3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:14.966823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ef4-5c93-4ad9-a577-c834378a50c3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:14.967114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e64ef4-5c93-4ad9-a577-c834378a50c3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:14.968615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 499/1012] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:14.973699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a6a0-1ab4-4514-b76b-fd063ecd718a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:14.974148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a6a0-1ab4-4514-b76b-fd063ecd718a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:14.974244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a6a0-1ab4-4514-b76b-fd063ecd718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:14.992196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fbd3a6a0-1ab4-4514-b76b-fd063ecd718a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:14.993387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fbd3a6a0-1ab4-4514-b76b-fd063ecd718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:15.061310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 514/1013] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:14 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:15.335387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64ef4-5c93-4ad9-a577-c834378a50c3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:15.352588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c9c60-ea90-46cb-8d82-1d2effcd2af5 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.352870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c9c60-ea90-46cb-8d82-1d2effcd2af5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.353142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c9c60-ea90-46cb-8d82-1d2effcd2af5 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.354481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 515/1014] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:15.359308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.359590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.359862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.378496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:15.379693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:15.400727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:15.453044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 516/1015] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:15.463987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e64ef4-5c93-4ad9-a577-c834378a50c3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:15.499120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:15.499738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.500023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.500313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.500889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:15.502245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:15.510975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:15.512035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:15.514083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 500/1016] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:15.521080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:15.624331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b0eda8-56f3-4aa9-9205-227dee2515b5 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:15.658124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:15.658727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.659003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.659274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.659888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:15.660187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:15.661573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:15.666477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 517/1017] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:15.672529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:15.697327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:15.727902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:15.753127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:15.777887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89cd0-3b8c-4e89-a4ae-9dd417ce96ff None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:15.815417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:15.816378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.816655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.817221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.817859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:15.818154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:15.819683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:15.829540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 501/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Thu Sep 22 12:17:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:15.830837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:15.838898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401fff3e-9ddd-4593-9616-3f80e57215f4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:15.877744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:15.877744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.877957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.878172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.878818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:15.880237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8f4cc3-5cce-4003-8491-ddefcf0d5fc3 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:15.898586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 518/1019] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:17:15 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:17:15.930972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:15.949197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088b8f4-4c6c-4597-901d-6c43d8d39e9b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:15.992503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:15.993115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:15.993385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:15.993650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:15.994302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:15.995655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55303c62-15ae-4ba2-9a5f-97897ed91e78 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:16.024338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 502/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:16.846591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c350c0d-7852-4219-9b4f-69f9b4da27b8 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:16.847111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c350c0d-7852-4219-9b4f-69f9b4da27b8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:16.847111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c350c0d-7852-4219-9b4f-69f9b4da27b8 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:16.848645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 519/1021] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:16.854176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:16.854586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:16.854737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:16.877445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:16.878632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:16.951824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 503/1022] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:17.220074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c350c0d-7852-4219-9b4f-69f9b4da27b8 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:17.300562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:17.322425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c350c0d-7852-4219-9b4f-69f9b4da27b8 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:17.355318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:17.355945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:17.356217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:17.357043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:17.358142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:17.359556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40fbb6-06dd-427e-ab78-77f8c01dc417 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:17.366134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 520/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:17.367394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:17.429796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:17.470201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08c0a2-5c29-4f36-8dd8-5c353863fd52 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:17.504819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:17.505425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:17.505701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:17.505972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:17.506611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:17.508037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18c957-73e6-4f70-a667-75b95f8f10b7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:17.540321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 504/1024] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:17:17 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:18.120891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583ff1-167c-42cf-a4d8-a65af25bb56f None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.121406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583ff1-167c-42cf-a4d8-a65af25bb56f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.121406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583ff1-167c-42cf-a4d8-a65af25bb56f None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.122900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 521/1025] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:18.128220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f4796-efec-4c82-9188-0feb1986caaf None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.128679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f4796-efec-4c82-9188-0feb1986caaf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.128851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2f4796-efec-4c82-9188-0feb1986caaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.147676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3a2f4796-efec-4c82-9188-0feb1986caaf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:18.148895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3a2f4796-efec-4c82-9188-0feb1986caaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:18.223329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 505/1026] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:18.313625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461cc7-c71a-402c-917c-32626a5910af None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.313975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461cc7-c71a-402c-917c-32626a5910af None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.314170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2461cc7-c71a-402c-917c-32626a5910af None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.315634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 522/1027] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:18.320762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.321049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.321323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.339085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:18.340242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:18.386589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:18.410470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 506/1028] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:18.520233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2461cc7-c71a-402c-917c-32626a5910af None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:18.621505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2461cc7-c71a-402c-917c-32626a5910af None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:18.654823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:18.655376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.655652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.655917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] PATH_INFO: `/v3/services/image` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.656503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:18.657861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:18.666413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:18.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:18.668582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 523/1029] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 22 12:17:18 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:18.673512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:18.679655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:18.693973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:18.785816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25724b6-a83f-404a-a537-5937615c673b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:18.786018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14dca7-2d57-4d19-9973-97a50aa72e18 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:18.822044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:18.822533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:18.822685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.822946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.823217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.823325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.823495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.823768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.824417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] RBAC: Adding query filter params (name=image) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:18.824723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:18.826627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:18.831673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 507/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:18.837023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:18.839053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:18.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:18.846515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:18.847378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:18.848783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 524/1031] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:17:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:18.853455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:18.893929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:18.929918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:18.952798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a5f527-7fea-4a80-b5e0-506e397ddc93 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:18.962003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa58c-74be-4a3b-b58a-a9b8b341fd96 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:18.986905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:18.987502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.987779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.988053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:18.988692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] RBAC: Adding query filter params (type=image) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:18.988991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:18.990339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:18.995365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 508/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:18.998639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:18.999232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:18.999503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:18.999764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.000678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:19.000795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.001470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.002175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.015733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 525/1033] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:19.016857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.102946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1004bd-ac56-43d4-8753-e0bce6c6f87f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.112995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:19.122492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f59c-cdd8-4eba-bbf3-e39e6e73c538 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.143670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.144616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.144897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.145169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.145855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] RBAC: Adding query filter params (interface=public, service_id=b51caeef39c5431fb3bb450b7846c7e9, region_id=RegionOne) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:19.146160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.147521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.157996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 509/1034] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/endpoints?service_id=b51caeef39c5431fb3bb450b7846c7e9&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:19.158924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.161278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.161963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.162233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.162502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.164505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:19.174211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.175672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:19.190113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:19.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:19.191985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:19.191985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:19.191985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:19.191985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 526/1035] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:19.196762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.271731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e93284b-c1ed-4e38-99be-2c9c3601117c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.305316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.305917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.306024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a9cf-23fe-47ba-bc93-a5b61038f167 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.306199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.306444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.307028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.308829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.320339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 510/1036] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:17:19.343111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.343924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.344207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.344511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.345173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:19.345475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.346886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.365260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 527/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:19.367154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.407408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:19.478730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648872-917f-4e31-964d-581d78ce5eb7 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.515507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.516133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.516437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.516704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.523789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.525348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:19.536681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:19.537645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:19.538608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 511/1038] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:19.543572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.592046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:19.646618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acc39c-3e16-46f2-942c-f481f3d93952 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.685308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.685915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.686189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.686458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.687176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:19.687476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.688963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35abcb0a-762f-43c7-83a1-622ae88221f4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:19.711002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 528/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:19.712666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:19.773248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f716c5-df27-4f0b-8d02-bf6653c40ed2 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.773443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f716c5-df27-4f0b-8d02-bf6653c40ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.773683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f716c5-df27-4f0b-8d02-bf6653c40ed2 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.775020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 529/1040] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:19.780595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.780984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.781255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.800030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:19.801246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:19.817540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:19.823713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a755ab6-fec5-4bb4-ab21-413231f179a2 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:19.862853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:19.863464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:19.863741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:19.864007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:19.880864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 530/1041] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:19.934181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:19.935685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.026278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.039168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 512/1042] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:20.040447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.074842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.141789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea946579-6ca3-481e-9625-df47cc431513 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.155432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.179780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.180436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.180709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.180978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.190807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.192333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:20.208237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:20.209298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:20.210189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:20.210189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:20.210189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:20.210189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 531/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:20.216445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.267433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec7e9b-f7e1-42fb-95d5-3e50eeb17d04 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.303072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.303781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.304060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.304746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.308159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.316322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:20.321041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:20.321967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:20.323731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 513/1044] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:20.327004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73986-2154-47ff-9a07-0f1e2554721e None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.328556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.363936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.364934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.364934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.365108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.365671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:20.366024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.367369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.379361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 532/1045] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:20.384598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.386406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.435195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53dc7ad-6ade-470c-a0f7-a0152b755ea6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.473645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.474249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.474523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.475049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.475691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:20.475988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.477362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.485193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67d30d-2c5c-488c-8e3b-8fc22a1b62ac None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.490669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 514/1046] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:20.491501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.526973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.527600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.527864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.528158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.535474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.537601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:20.548773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:20.549848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:20.550780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 533/1047] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:20.555478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.564777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.603751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9c50-131a-4fa0-8017-01c669d1b953 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.621705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.640359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.640982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.641253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.641521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] PATH_INFO: `/v3/users/ccd8c8e8d8884c6092fc8185a6646287` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.655742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] RBAC: Authorizing `identity:get_user(user_id=ccd8c8e8d8884c6092fc8185a6646287)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.657233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.658997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 515/1048] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/users/ccd8c8e8d8884c6092fc8185a6646287 => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:20.664401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.667943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230f1de-d34a-42a3-94f9-0f72ca137b50 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.704155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.705191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.705467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.705741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.706756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:20.707067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.708858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.721001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 534/1049] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:20.726968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:20.767937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79daf3-91e1-4c92-b6e7-03eee187b750 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.792974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.802899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.803493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.803771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.804033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.831042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbef092-6448-4fa2-a598-9e275700513c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:20.851218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:20.871741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748929-4785-4812-850c-bf9e35593194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:20.873037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748929-4785-4812-850c-bf9e35593194 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:20.873648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748929-4785-4812-850c-bf9e35593194 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:20.873648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748929-4785-4812-850c-bf9e35593194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:20.874344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748929-4785-4812-850c-bf9e35593194 None admin] RBAC: Adding query filter params (role.id=036a2147287c4f9eb1fc966fb2da43df, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:20.874707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748929-4785-4812-850c-bf9e35593194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.876181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748929-4785-4812-850c-bf9e35593194 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:20.881703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:20.883002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 535/1050] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=036a2147287c4f9eb1fc966fb2da43df => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:20.883174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:21.000685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 516/1051] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 12:17:21.004436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748929-4785-4812-850c-bf9e35593194 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:21.111787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748929-4785-4812-850c-bf9e35593194 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:21.140593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:21.147284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:21.147931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:21.148175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:21.148469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:21.149562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:21.150982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783bd22-1b3b-4283-bc07-ea81b8af9d9e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:21.152782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 536/1052] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:21 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:21.158991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:21.237617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:21.262025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe996a-4e68-4cb3-8ab9-80312f58b6fd None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:21.296540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:21.297138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:21.297411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:21.297677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:21.298377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f, user.id=ccd8c8e8d8884c6092fc8185a6646287) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:21.298677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:21.300075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecef71-2730-4d3b-ba59-b0199d800dc6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:21.308450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 517/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:21 2022] GET /identity/v3/role_assignments?user.id=ccd8c8e8d8884c6092fc8185a6646287&scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:21.519233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b198f-2c63-46b7-b1a1-30b580babd38 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:21.519534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b198f-2c63-46b7-b1a1-30b580babd38 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:21.519714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b198f-2c63-46b7-b1a1-30b580babd38 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:21.521205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 537/1054] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:21.526218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:21.526509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:21.526788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:21.545699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:21.546877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:21.616035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 518/1055] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:21.897749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b198f-2c63-46b7-b1a1-30b580babd38 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:22.002889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b198f-2c63-46b7-b1a1-30b580babd38 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:22.036855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:22.037487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:22.037762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:22.038031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:22.038599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:22.039958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:22.049254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:22.050246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 12:17:22.051093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:22.055361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 538/1056] 10.222.0.33 () {56 vars in 1105 bytes} [Thu Sep 22 12:17:21 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:22.056635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:22.145514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:22.191799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e31d-cb08-48a2-8f58-f2c984330c44 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:22.227529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:22.228134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:22.228452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:22.228726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:22.229345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] RBAC: Adding query filter params (name=image) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:22.229645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:22.231021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:22.236772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 519/1057] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:22.242236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:22.305401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:22.345814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88537a0-3399-4b69-9ea4-05919e24f0a8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:22.381713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:22.382871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:22.383205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:22.383480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:22.384104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] RBAC: Adding query filter params (type=image) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:22.384442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:22.385884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80daef7-5201-4ac3-bae3-176397bf5d2a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:22.396936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 539/1058] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:22.398192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:22.509102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379de524-e344-48fb-b1db-cc3b9db79907 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:22.546067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:22.546682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:22.546965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:22.547241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:22.547899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:22.549351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011f20d-1324-412c-b73c-c100938f5eba None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:22.553881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:22.580332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 520/1059] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:17:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:23.065847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a23ee-e363-4ef9-a734-d81e9bee723e None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.066222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a23ee-e363-4ef9-a734-d81e9bee723e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.066423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a23ee-e363-4ef9-a734-d81e9bee723e None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.067915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 540/1060] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:23.072792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59f0fb-c223-4137-8145-6c8806241737 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.073083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59f0fb-c223-4137-8145-6c8806241737 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.073361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59f0fb-c223-4137-8145-6c8806241737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.091890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3d59f0fb-c223-4137-8145-6c8806241737 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:23.093125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3d59f0fb-c223-4137-8145-6c8806241737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:23.172560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 521/1061] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:23.437033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a23ee-e363-4ef9-a734-d81e9bee723e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:23.453845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51610034-2e1e-4781-84cb-acf3ff8fdf63 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.454315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51610034-2e1e-4781-84cb-acf3ff8fdf63 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.454417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51610034-2e1e-4781-84cb-acf3ff8fdf63 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.455731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 522/1062] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:23.460888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.461158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.461437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.481243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:23.482455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:23.549215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a23ee-e363-4ef9-a734-d81e9bee723e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:23.554103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 523/1063] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:23.579565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:23.588944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:23.589576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.589853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.590125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.601013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:23.605744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:23.606712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:23.607592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:23.611912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 541/1064] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:17:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:23.616341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:23.658629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:23.712766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd779e5-3833-4a57-a8a7-249e885c38ea None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:23.745361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:23.745954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.746227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.746494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.747670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:23.747964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:23.749327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:23.757207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 524/1065] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:23.763310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:23.828710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:23.880971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb601968-c988-4ca1-a5f1-eef64c7a070c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:23.917488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:23.918466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.919271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.919540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.931269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:23.932741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:23.943980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf34e0-b3ad-4eb2-a2ee-287131aab7d0 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:23.948159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:23.954203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:23.955578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:23.955578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:23.955578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:23.955578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 542/1066] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:23.955830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:23.956310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:23.978982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:23.979533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:23.979808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:23.980075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:23.991374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:23.996075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:23.997683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:23.999466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 525/1067] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:24.001628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.004370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.061809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82fb3a8-4bc0-498d-a042-65ebad741f95 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.096240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.096867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.097144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.097410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.098029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:24.098428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.100171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.108707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821aa2c-21d7-4c75-ac7a-791f2ec85eba None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.117599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 543/1068] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:24.118576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.145715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.146310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.146582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.146848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.147569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:24.147782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.149162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.161724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 526/1069] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:24.162727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.193469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.224014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.228655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5185c-9f1c-469e-882a-1af21b2ed936 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.263594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.264153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.264458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.264729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.267874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb99f69-e3bd-4190-ac13-16ba5702452c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.271961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.273432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:24.285477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:24.286464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:24.287410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 544/1070] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:24.292148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.307599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.308229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.308540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.308810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] PATH_INFO: `/v3/users/ccd8c8e8d8884c6092fc8185a6646287` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.309943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] RBAC: Authorizing `identity:get_user(user_id=ccd8c8e8d8884c6092fc8185a6646287)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.311799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.313583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 527/1071] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/users/ccd8c8e8d8884c6092fc8185a6646287 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:24.318902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.399268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f531c-388b-46b8-b72e-b627515498d7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.431314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.435195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea17a6c-15cf-429b-a305-9410dbdefa7e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.436029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.436664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.437047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.437583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.437913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:24.438209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.439633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.457593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 545/1072] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:24.463721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.476994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.477899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.478170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.478438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.479534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.480982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.483377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.483619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 528/1073] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:24.489015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.583745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8eca-fccf-40ed-b759-e04e38eb8596 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.604008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d8342a-934f-4fce-9cc8-44bbab740d0d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.622783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.623406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.623674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.623961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.634230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.635650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.639507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.640160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.640468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.640739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f/users/ccd8c8e8d8884c6092fc8185a6646287/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.642626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8dea0eaa91644a998079bed4a9d4a6f, user_id=ccd8c8e8d8884c6092fc8185a6646287, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.644484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac8df9-00f0-4b37-897c-a58144b1f425 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:24.652459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:24.653479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:24.654481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:24.654481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:24.654481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:24.658765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 546/1074] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:24.660162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.662888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 529/1075] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:17:24 2022] PUT /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f/users/ccd8c8e8d8884c6092fc8185a6646287/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:17:24.739481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.777341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66cebf-cfff-4a76-988d-e1125ae85554 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.789978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f56ae-685b-4123-a831-3ab1d3d6a27f None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.790333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f56ae-685b-4123-a831-3ab1d3d6a27f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.790542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f56ae-685b-4123-a831-3ab1d3d6a27f None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.792075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 530/1076] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:24.798101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.798393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.798651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.817803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:24.818053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.818466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.818729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.818997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.819099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:24.819638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:24.819936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.821357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:24.834906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 547/1077] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:24.840580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:24.846738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.902624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 531/1078] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:24.946773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa3804-f7cb-450e-907f-bc0e12bb50fd None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:24.973381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:24.982660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:24.983285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:24.983561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:24.983827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:24.990590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:24.992056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:25.003763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:25.004872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:25.005796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 548/1079] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:25.010700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:25.113291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cf63-e3ec-45dc-9da7-b04ca47f1a57 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:25.147797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:25.148435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:25.148715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:25.148985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:25.149659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:25.149958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:25.151361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.163023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 532/1080] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:25.168923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:25.173499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:25.192032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:25.279592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3bb33c-9bb3-4a27-89b7-4a6dece238a8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:25.280949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c653-b81f-4ab8-a70c-ffa8fd06cb48 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:25.314564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:25.315162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:25.315435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:25.315704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:25.316944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:25.317784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:25.318328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:25.318880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/036a2147287c4f9eb1fc966fb2da43df` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:25.320985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=6db6f737c3354ca692eb6ee91a496223, role_id=036a2147287c4f9eb1fc966fb2da43df)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:25.322403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.348332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 549/1081] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:17:25 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/6db6f737c3354ca692eb6ee91a496223/roles/036a2147287c4f9eb1fc966fb2da43df => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:17:25.402019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:25.403457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.423190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:25.519296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 533/1082] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:25.526545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:25.579957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:25.633255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde9a424-7a26-4f9b-a1b5-4e82319cb589 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:25.667054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:25.667603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:25.667877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:25.668146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:25.678609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:25.680043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d999de82-0281-46b3-be48-0694129e8b8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:25.693824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:25.694785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:25.695699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:25.695699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:25.695699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:25.695699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 550/1083] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:25.701396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:25.798962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1f51a-d87c-4d08-aba5-e8c145c3f457 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:25.815833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:25.835972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:25.836613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:25.836893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:25.837162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:25.837790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:25.838088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:25.839439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75adfa9-a5bd-4bd3-9edc-cec2c7221585 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:25.851087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 534/1084] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:26.805729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbb754-0305-44e2-bd29-c60f2b7b147b None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:26.806290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbb754-0305-44e2-bd29-c60f2b7b147b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:26.806290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cbb754-0305-44e2-bd29-c60f2b7b147b None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:26.807584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 551/1085] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:26.812931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:26.813329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:26.813475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:26.830997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:26.832150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:26.900255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 535/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:27.169460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbb754-0305-44e2-bd29-c60f2b7b147b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.267391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:27.271567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cbb754-0305-44e2-bd29-c60f2b7b147b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:27.302855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:27.303464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.303733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.303996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.314459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:27.319103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:27.320193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:27.321172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 552/1087] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:27.326711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.422968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8914dd90-3b51-4fd2-a542-7292f10269c4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:27.431025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:27.458174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:27.458774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.459050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.459320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.459966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:27.460266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:27.461647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f08155-89ef-4d19-b36b-f500136c3074 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:27.469367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 536/1088] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:27.475487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.520899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d64d-ef95-4543-a0fe-eb6f8c6757e8 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.521183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d64d-ef95-4543-a0fe-eb6f8c6757e8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.521468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d64d-ef95-4543-a0fe-eb6f8c6757e8 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.522817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 537/1089] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:17:27.527868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.528150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.528450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.547687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:27.548909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:27.582735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc6e75-88d5-42a2-abab-80138693c30f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:27.614987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:27.615609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.615886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.616158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] PATH_INFO: `/v3/users/ccd8c8e8d8884c6092fc8185a6646287` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.617269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 538/1090] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:17:27.617411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] RBAC: Authorizing `identity:get_user(user_id=ccd8c8e8d8884c6092fc8185a6646287)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:27.618710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:27.625283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 553/1091] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/users/ccd8c8e8d8884c6092fc8185a6646287 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:27.626344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.716459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:27.720109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7114931-6caf-4b2a-a760-4cb61b95b929 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:27.753177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:27.753762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:27.754033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:27.754298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:27.816887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:27.818311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:27.884934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.912569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:27.912727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:27.918029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 539/1092] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:27.924415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:27.986032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa43df-29c5-49d8-811f-1b55ac8e3b83 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.018542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.019162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.019444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.019709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.021057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630f67-f15c-4e0f-80b4-cb5b696c41f1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.030307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:28.035030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:28.036136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:28.037390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 554/1093] 10.222.0.33 () {56 vars in 1120 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:28.041995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.058076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.058684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.058981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.059259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.060385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.061809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.063552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 540/1094] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:27 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:28.069354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.126052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:28.131316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:28.138722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146fa44-151f-4c6d-bc1a-962151fc8059 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.165280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772887da-1c70-4f1b-ae03-3c30620ae63f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.172511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.173114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.173391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.173661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.174309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:28.174609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.175965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.183757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 555/1095] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:28.189612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.197364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.197950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.198220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.198484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.199179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f, user.id=ccd8c8e8d8884c6092fc8185a6646287) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:28.199476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.200939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.206751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 541/1096] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/role_assignments?user.id=ccd8c8e8d8884c6092fc8185a6646287&scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:28.298743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff83e848-c0c3-495c-8462-23ca39dae58f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.332571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.333210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.333483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.333748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.347012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.348504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:28.364197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:28.365224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:28.366131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:28.366131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:28.366131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:28.366131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 556/1097] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:28.371146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.472176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99929c25-f775-472a-bad3-fc33c531765e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.502055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:28.508418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.509026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.509305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.509579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.510213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:28.510510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.511880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.524179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 542/1098] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:28.530735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.533650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:28.628509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b0b67-ee21-4079-9cf5-baea0d842ba7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.663163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.663814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.664090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.664396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.674946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.676494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:17:28.687338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:28.688423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:28.689734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 557/1099] 10.222.0.33 () {56 vars in 1093 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:28.694154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.791154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea561408-161e-4add-87ff-d8239025f86c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.823737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.824356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.825209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.825482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:28.826180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:28.826476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:28.827875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:28.844140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 543/1100] 10.222.0.33 () {56 vars in 1133 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:28.850156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:28.912102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:28.950528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a74ba9-be30-4647-9cea-9b766778a3d8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:28.983528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:28.984131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:28.984444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:28.984716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.049722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.051167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.108702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:29.150457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 558/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 12:17:29.156210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:29.203085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:29.252112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe864b-66d8-40c2-b452-9a49a885d7f8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:29.283798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:29.284432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:29.284705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:29.284967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.295056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.296505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:29.310969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:29.316594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:17:29.317556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:29.317556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:17:29.317556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:29.317556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 544/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:17:29.317792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:29.412481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bdc077-0a93-4b8f-954b-ec946b978728 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:29.446090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:29.446694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:29.446987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:29.447234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.447855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:29.448149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.449549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.460689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 559/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:29.466222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:29.561798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:29.564950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d321ce0d-4dd2-479e-9e70-302b1561e11c None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:29.597085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2c883-b000-4288-aa73-e357714084af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:29.597695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2c883-b000-4288-aa73-e357714084af None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:29.597968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2c883-b000-4288-aa73-e357714084af None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:29.598234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b2c883-b000-4288-aa73-e357714084af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.604792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2c883-b000-4288-aa73-e357714084af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.606785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b2c883-b000-4288-aa73-e357714084af None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-21b2c883-b000-4288-aa73-e357714084af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:17:29.617489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:29.618561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:29.619479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 545/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:17:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:29.624368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:29.648461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:29.723829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f2bac-ed64-46c8-a8de-ca3aa3bc1323 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:29.758174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:29.758772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:29.759049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:29.759320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.759995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:29.760311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.761757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee4044-e26c-436b-8335-540fed502122 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.773324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 560/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:17:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:29.780045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2c883-b000-4288-aa73-e357714084af None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:29.878358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b2c883-b000-4288-aa73-e357714084af None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:29.911780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:29.912407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:29.912686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:29.912955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:29.913662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] RBAC: Adding query filter params (role.id=036a2147287c4f9eb1fc966fb2da43df, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=6db6f737c3354ca692eb6ee91a496223) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:29.913966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:29.915367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eee308-446a-4931-b809-96a984c489f6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:29.921183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 546/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:29 2022] GET /identity/v3/role_assignments?user.id=6db6f737c3354ca692eb6ee91a496223&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=036a2147287c4f9eb1fc966fb2da43df => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:30.390621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a432-745f-40c6-ae6f-5beaf0ad4b5f None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:30.391134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a432-745f-40c6-ae6f-5beaf0ad4b5f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:30.391238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5a432-745f-40c6-ae6f-5beaf0ad4b5f None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:30.392681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 561/1107] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:30.398143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:30.398423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:30.398698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:30.417107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:30.418283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:30.438519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:30.492830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 547/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:30.775087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5a432-745f-40c6-ae6f-5beaf0ad4b5f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:30.854102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:30.873088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5a432-745f-40c6-ae6f-5beaf0ad4b5f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:30.906152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:30.906761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:30.907032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:30.907297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:30.908401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:30.909813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ca5cc-6ce7-4b5b-b1a1-0be7d6aecf19 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:30.911342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 562/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:30.917196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:31.011207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878597a-3c81-473c-96bb-a0478e328621 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:31.043601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:31.044126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:31.044444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:31.044717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:31.045360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:31.046760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ac093-a02f-46fe-ab1a-14c62906f0bf None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:31.066769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 548/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:17:30 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:33.236648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd4825-9a05-41a8-837a-f0c3cdd08c90 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:33.237144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd4825-9a05-41a8-837a-f0c3cdd08c90 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:33.237144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd4825-9a05-41a8-837a-f0c3cdd08c90 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:33.238408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 563/1111] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:33.243662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:33.244079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:33.244197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:33.262730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:33.263887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:33.266531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:33.329410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 549/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:33.601065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd4825-9a05-41a8-837a-f0c3cdd08c90 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:33.699102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd4825-9a05-41a8-837a-f0c3cdd08c90 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:33.732194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:33.732842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:33.733120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:33.733392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] PATH_INFO: `/v3/roles/49615db920c64505958165b68049a95e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:33.734775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] RBAC: Authorizing `identity:get_role(role_id=49615db920c64505958165b68049a95e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:33.736124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:33.737470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 564/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:33 2022] GET /identity/v3/roles/49615db920c64505958165b68049a95e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:33.743174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:33.838799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7d253-408b-47e3-bcfc-12e00c28ad4d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:33.871960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:33.872551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:33.873188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:33.873188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] PATH_INFO: `/v3/users/785e36f8233e4ebda7ba75573cbd99ae` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:33.883092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:33.886609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] RBAC: Authorizing `identity:get_user(user_id=785e36f8233e4ebda7ba75573cbd99ae)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:33.888068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:33.889708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 550/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:33 2022] GET /identity/v3/users/785e36f8233e4ebda7ba75573cbd99ae => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:33.898016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:33.916436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:33.994427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b53c61-a5ee-4b48-87dd-f31195ea55ea None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:34.028422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:34.029007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:34.029278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:34.029543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:34.092011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:34.093466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:34.110470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:34.186568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 565/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:34.192403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:34.286427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898b9d0-7940-4e70-b265-1be90e80647d None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:34.318550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:34.319147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:34.319418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:34.319683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:34.320776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:34.322174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76d38a-8662-402d-9794-e41923cd1022 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:34.323738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 551/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:34 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:34.329693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:34.424769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32812f2b-2016-4dfb-9f8f-cdd954a9231b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:34.441775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:34.458751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:34.459339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:34.459612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:34.459880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:34.460616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f, user.id=785e36f8233e4ebda7ba75573cbd99ae) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:34.460919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:34.462317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aebd20-d56e-4fa0-acfd-d564b6618c83 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:34.468076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 566/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:34 2022] GET /identity/v3/role_assignments?user.id=785e36f8233e4ebda7ba75573cbd99ae&scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f&role.id=49615db920c64505958165b68049a95e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:36.566701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c859f-aa3b-4151-8b32-5db2d7e88b88 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:36.567261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c859f-aa3b-4151-8b32-5db2d7e88b88 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:36.567261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c859f-aa3b-4151-8b32-5db2d7e88b88 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:36.568587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 552/1118] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:36.573774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:36.574137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:36.574317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:36.592372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:36.593528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:36.656389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 567/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:36.924356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c859f-aa3b-4151-8b32-5db2d7e88b88 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:36.988249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:37.024968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c859f-aa3b-4151-8b32-5db2d7e88b88 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:37.059291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:37.059970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:37.060247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:37.060552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] PATH_INFO: `/v3/roles/49615db920c64505958165b68049a95e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:37.061939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] RBAC: Authorizing `identity:get_role(role_id=49615db920c64505958165b68049a95e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:37.063278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:37.064624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 553/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:36 2022] GET /identity/v3/roles/49615db920c64505958165b68049a95e => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:37.070353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:37.163720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b66e0-e7c5-429c-88e8-95bbd7fa78ae None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:37.176513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:37.196478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:37.197089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:37.197361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:37.197625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] PATH_INFO: `/v3/users/785e36f8233e4ebda7ba75573cbd99ae` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:37.198705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=785e36f8233e4ebda7ba75573cbd99ae)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:37.200110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:37.201672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 568/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:37 2022] GET /identity/v3/users/785e36f8233e4ebda7ba75573cbd99ae => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:37.207240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:37.304140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a6f2c-4c4a-41d0-8d01-53aa6583641f None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:37.335071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:37.335675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:37.335947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:37.336211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:37.337312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:37.338717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6ae46-43f7-4824-a232-b29684b94960 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:37.340311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 554/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:37 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:37.346645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:37.441798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084538e5-d032-4fb5-8253-428bb8fbb2f5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:37.474641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:37.475271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:37.475546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:37.475815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f/users/785e36f8233e4ebda7ba75573cbd99ae/roles/49615db920c64505958165b68049a95e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:37.477928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8dea0eaa91644a998079bed4a9d4a6f, user_id=785e36f8233e4ebda7ba75573cbd99ae, role_id=49615db920c64505958165b68049a95e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:37.479345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5273d4d-7391-408f-b955-d158a41133b6 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:37.491589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 569/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:17:37 2022] PUT /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f/users/785e36f8233e4ebda7ba75573cbd99ae/roles/49615db920c64505958165b68049a95e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:17:39.614726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86b1e-60fe-494d-ab26-fad59138d7ce None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:39.615281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86b1e-60fe-494d-ab26-fad59138d7ce None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:39.615281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef86b1e-60fe-494d-ab26-fad59138d7ce None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:39.616597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 555/1124] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:39.621672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26738-27f3-415a-b466-25561dab48cb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:39.622095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26738-27f3-415a-b466-25561dab48cb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:39.622198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c26738-27f3-415a-b466-25561dab48cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:39.640366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-20c26738-27f3-415a-b466-25561dab48cb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:39.641541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-20c26738-27f3-415a-b466-25561dab48cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:39.674453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:39.707346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 570/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:39.974074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef86b1e-60fe-494d-ab26-fad59138d7ce None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:39.977192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:40.072629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef86b1e-60fe-494d-ab26-fad59138d7ce None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:40.104810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:40.105426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:40.105702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:40.105972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] PATH_INFO: `/v3/roles/49615db920c64505958165b68049a95e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:40.107338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] RBAC: Authorizing `identity:get_role(role_id=49615db920c64505958165b68049a95e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:40.108728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:40.110091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 556/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:39 2022] GET /identity/v3/roles/49615db920c64505958165b68049a95e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:40.115825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c26738-27f3-415a-b466-25561dab48cb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:40.214157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c26738-27f3-415a-b466-25561dab48cb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:40.246198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:40.246812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:40.247089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:40.247888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] PATH_INFO: `/v3/users/785e36f8233e4ebda7ba75573cbd99ae` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:40.249010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] RBAC: Authorizing `identity:get_user(user_id=785e36f8233e4ebda7ba75573cbd99ae)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:40.250426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:40.251980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 571/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:40 2022] GET /identity/v3/users/785e36f8233e4ebda7ba75573cbd99ae => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:40.257705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:40.323364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:40.353755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1cad4-e7b1-4369-84ba-efdde18da883 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:40.385595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:40.386169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:40.386441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:40.386706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:40.449117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:40.450549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:40.512964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:40.542010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 557/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:40.549119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:40.588313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:40.646114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098b454-4fa1-40f8-a57b-13fc6c57bac7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:40.679236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:40.679847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:40.680123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:40.680427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] PATH_INFO: `/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:40.681524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=a8dea0eaa91644a998079bed4a9d4a6f)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:40.682945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24493cc-49d6-4c87-957e-030bfaa4cbf5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:40.684578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 572/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:40 2022] GET /identity/v3/projects/a8dea0eaa91644a998079bed4a9d4a6f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:40.691206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:40.783972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53f1ba-a5b4-445a-a8e5-2da3c2eee5c1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:40.816348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:40.816929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:40.817201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:40.817468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:40.818162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] RBAC: Adding query filter params (role.id=49615db920c64505958165b68049a95e, scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f, user.id=785e36f8233e4ebda7ba75573cbd99ae) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:40.818460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:40.819854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e522c91-3376-4ee8-a3d5-acd5c4861a98 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:40.825414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 558/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:40 2022] GET /identity/v3/role_assignments?user.id=785e36f8233e4ebda7ba75573cbd99ae&scope.project.id=a8dea0eaa91644a998079bed4a9d4a6f&role.id=49615db920c64505958165b68049a95e => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:17:42.967695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb6b-4ae7-4a8a-8d36-a5042626db22 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:42.968211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb6b-4ae7-4a8a-8d36-a5042626db22 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:42.968211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9bb6b-4ae7-4a8a-8d36-a5042626db22 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:42.969614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 573/1131] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:42.975043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:42.975458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:42.975586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:42.993624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:42.994776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:43.020720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:43.058304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 559/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:43.323335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9bb6b-4ae7-4a8a-8d36-a5042626db22 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:43.422612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9bb6b-4ae7-4a8a-8d36-a5042626db22 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:43.429514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:43.455491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:43.456101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:43.456404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:43.456676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:43.457763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:43.459185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b3cac-f805-4981-95d9-21f567bf6410 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:43.460752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 574/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:43.467042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:43.560120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886f738-c34e-4901-8375-bbd7bcd0a57b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:43.592610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:43.593139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:43.593416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:43.593686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:43.594320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:43.595729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea33f0-9951-43db-bd5b-938d80bed3a7 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:43.620701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 560/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:17:43 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:17:45.757418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a31c8-0bce-4ea8-bda7-b8d3def92379 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:45.757947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a31c8-0bce-4ea8-bda7-b8d3def92379 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:45.757947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a31c8-0bce-4ea8-bda7-b8d3def92379 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:45.759276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 575/1135] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:45.765079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:45.765472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:45.765625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:45.786458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:45.787651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:45.799912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:45.855271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 561/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:46.129107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a31c8-0bce-4ea8-bda7-b8d3def92379 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:46.226147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a31c8-0bce-4ea8-bda7-b8d3def92379 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:46.258414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:46.259018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:46.259289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:46.259554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:46.260654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:46.262071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7663f-3d73-497b-8e59-a837027fed95 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:46.264124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 576/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:17:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:46.269678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:46.363120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf1ea0-9df3-4a17-a1f8-6533b92135eb None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:46.395112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:46.395639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:46.395912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:46.396180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:46.396850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:46.398442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1f445-3171-46db-80d4-6b420085689f None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:46.419432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 562/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:17:46 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:17:48.566388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564b23-69d6-4d99-8ace-467e9f67fda0 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:48.566944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564b23-69d6-4d99-8ace-467e9f67fda0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:48.566944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9564b23-69d6-4d99-8ace-467e9f67fda0 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:48.568236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 577/1139] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:48.573343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:48.573765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:48.573892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:48.580056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:48.594030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:48.595497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:48.661638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 563/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:48.927302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9564b23-69d6-4d99-8ace-467e9f67fda0 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:48.970633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:49.025512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9564b23-69d6-4d99-8ace-467e9f67fda0 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.057008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.057617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.057889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.058157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.068208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:49.072943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:49.074017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:49.074871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 578/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:49.080458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:49.173597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868a1be-f53f-4022-8642-2370eb331d6b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.202162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:49.207345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.207936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.208213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.208509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.209155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:49.209454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:49.210808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:49.218543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 564/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:49.224715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:49.320079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a4bec3-7a11-45a0-8994-07ef5088fcf8 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.321565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:49.354034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.354641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.354917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.355184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] PATH_INFO: `/v3/users/3c247c960d204a719da0c7f165d2d70b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.368064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] RBAC: Authorizing `identity:get_user(user_id=3c247c960d204a719da0c7f165d2d70b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:49.369513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:49.371067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 579/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:49 2022] GET /identity/v3/users/3c247c960d204a719da0c7f165d2d70b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:49.377258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:49.472624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63804c00-b795-499e-978d-cf96dc1044b9 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.505947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.506538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.506814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.507086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.564185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:49.571625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:49.573088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:49.665838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 565/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:49.672232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:49.767502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac106df4-e1bf-422a-a035-1f9a3cf94e15 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.800795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.801408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.801681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.801951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] PATH_INFO: `/v3/projects/348b92a329a041279e283ee3b0db4fc9` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.803038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] RBAC: Authorizing `identity:get_project(project_id=348b92a329a041279e283ee3b0db4fc9)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:49.804477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a338a4c-097a-4c3b-84ec-1952a8c503c5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:49.806062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 580/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:49 2022] GET /identity/v3/projects/348b92a329a041279e283ee3b0db4fc9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:49.812104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:49.899770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:49.907281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140b7bd-321e-428a-9946-3e6577e1f563 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:49.939741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:49.940340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:49.940616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:49.940883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:49.941578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=348b92a329a041279e283ee3b0db4fc9, user.id=3c247c960d204a719da0c7f165d2d70b) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:49.941878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:49.943273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d779ac-2470-4874-b44d-412d7e1118a4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:49.948426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 566/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:49 2022] GET /identity/v3/role_assignments?user.id=3c247c960d204a719da0c7f165d2d70b&scope.project.id=348b92a329a041279e283ee3b0db4fc9&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:52.023011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979232-5d92-4df1-9f24-3e10020a03f7 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.023550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979232-5d92-4df1-9f24-3e10020a03f7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.023550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e979232-5d92-4df1-9f24-3e10020a03f7 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.024870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 581/1147] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:52.030134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.030507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.030674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.048852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:52.050013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:52.113007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 567/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:52.378835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e979232-5d92-4df1-9f24-3e10020a03f7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:52.406405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:52.476666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e979232-5d92-4df1-9f24-3e10020a03f7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:52.507837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:52.508470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.508743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.509006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.519019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:52.523688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:52.524813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:52.525856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 582/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:52.531190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:52.624872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:52.636765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c424-233e-4a30-8624-a32c67cfd32d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:52.670029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:52.670619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.670896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.671167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.671814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:52.672115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:52.673508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:52.681228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 568/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:52.687011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:52.770977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:52.782847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442432e1-e045-445e-9447-41ee25571ea6 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:52.814668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:52.815263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.815532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.815792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] PATH_INFO: `/v3/users/3c247c960d204a719da0c7f165d2d70b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.816884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] RBAC: Authorizing `identity:get_user(user_id=3c247c960d204a719da0c7f165d2d70b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:52.818285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:52.819797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 583/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:52 2022] GET /identity/v3/users/3c247c960d204a719da0c7f165d2d70b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:52.826065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:52.926824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e510af-99e4-4bac-9353-03b4f2919e02 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:52.959153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:52.959761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:52.960033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:52.960317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] PATH_INFO: `/v3/projects/348b92a329a041279e283ee3b0db4fc9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:52.961397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] RBAC: Authorizing `identity:get_project(project_id=348b92a329a041279e283ee3b0db4fc9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:52.962812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957dea6-855d-4ad5-8b66-96c99e329416 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:52.964428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 569/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:52 2022] GET /identity/v3/projects/348b92a329a041279e283ee3b0db4fc9 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:52.970366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:53.064997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d367754-803f-4ba6-984d-6e746837919d None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:53.096908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:53.097538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:53.097807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:53.098070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] PATH_INFO: `/v3/projects/348b92a329a041279e283ee3b0db4fc9/users/3c247c960d204a719da0c7f165d2d70b/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:53.100164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=348b92a329a041279e283ee3b0db4fc9, user_id=3c247c960d204a719da0c7f165d2d70b, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:53.101596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c238f-62e4-4fd0-9ccb-5fa6c556f2a9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:53.114852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 584/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:17:52 2022] PUT /identity/v3/projects/348b92a329a041279e283ee3b0db4fc9/users/3c247c960d204a719da0c7f165d2d70b/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:17:55.199775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cd87-ea01-4bff-a0d6-058e358171c4 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:55.200347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cd87-ea01-4bff-a0d6-058e358171c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:55.200347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528cd87-ea01-4bff-a0d6-058e358171c4 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:55.201576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 570/1154] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:55.206679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:55.207105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:55.207203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:55.225547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:55.226713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:55.243051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:55.308669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 585/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:55.573231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528cd87-ea01-4bff-a0d6-058e358171c4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:55.592505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:55.670738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528cd87-ea01-4bff-a0d6-058e358171c4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:55.702412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:55.703010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:55.703280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:55.703542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:55.713066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:17:55.718337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:55.719508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:17:55.720636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 571/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:17:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:55.725557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:55.819911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034c77f-adf6-4bcb-ae96-2c62785b8485 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:55.852408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:55.852943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:55.853220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:55.853491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:55.854134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:55.854432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:55.855790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:55.863949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 586/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:17:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:55.869829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:55.944631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:55.965663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cd97-7519-45ef-9f16-1b3c0993b9d3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:55.997777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:55.998378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:55.998650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:55.998916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] PATH_INFO: `/v3/users/3c247c960d204a719da0c7f165d2d70b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:55.999992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] RBAC: Authorizing `identity:get_user(user_id=3c247c960d204a719da0c7f165d2d70b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:56.001421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:56.002942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 572/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:17:55 2022] GET /identity/v3/users/3c247c960d204a719da0c7f165d2d70b => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:56.008646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:56.020701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:56.106536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45f1de-6d2f-4d93-b1c9-e3060dbebd83 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:56.138483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:56.139075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:56.139349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:56.139613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:56.204027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:56.205475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:56.214421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:56.298772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 587/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:56.304790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:56.400953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12bc64-e85c-458b-967a-a01f23155763 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:56.433073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:56.433682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:56.434544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:56.434824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] PATH_INFO: `/v3/projects/348b92a329a041279e283ee3b0db4fc9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:56.435910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] RBAC: Authorizing `identity:get_project(project_id=348b92a329a041279e283ee3b0db4fc9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:56.437364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f856e0-7d96-464a-9188-e13a9b1b2cae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:56.438946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 573/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:17:56 2022] GET /identity/v3/projects/348b92a329a041279e283ee3b0db4fc9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:56.444763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:56.539223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3983f71-6dd6-46af-9956-0fe1481d5474 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:56.548584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:56.572775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:56.573362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:56.573639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:56.573912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:56.574640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=348b92a329a041279e283ee3b0db4fc9, user.id=3c247c960d204a719da0c7f165d2d70b) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:56.574948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:56.576382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269267db-4056-404e-a0a4-a5ce8eaf23a2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:56.581899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 588/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:17:56 2022] GET /identity/v3/role_assignments?user.id=3c247c960d204a719da0c7f165d2d70b&scope.project.id=348b92a329a041279e283ee3b0db4fc9&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:17:58.707399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123104a-2bf6-495d-820d-21904a88a531 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:58.707960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123104a-2bf6-495d-820d-21904a88a531 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:58.707960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123104a-2bf6-495d-820d-21904a88a531 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:58.709284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 574/1162] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:17:58.714499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:58.714910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:58.715558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:58.734015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:17:58.735183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:17:58.797985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 589/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:17:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:17:59.062332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123104a-2bf6-495d-820d-21904a88a531 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:59.078061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:59.162265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123104a-2bf6-495d-820d-21904a88a531 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:59.193862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:59.194451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:59.194728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:59.194998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:59.259292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:59.260761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:59.274609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:59.355061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 575/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:17:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:17:59.360985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:59.454823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0923ef0-46a9-4ba0-9a03-868afd007d7b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:59.456266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:59.488628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:59.489234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:59.489506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:59.489772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:59.496067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:59.498048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-06541eeb-2068-46f1-9637-41226aa42e18 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:17:59.508962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:17:59.510178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:17:59.511272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 590/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:17:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:17:59.516567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:17:59.616019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9ad13-220d-48e1-8a8b-6a29d8715d60 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:17:59.629740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:17:59.649730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:17:59.650321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:17:59.650598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:17:59.650870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:17:59.651488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:17:59.651787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:17:59.653210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa73aac-d674-4df5-85c6-eabf1c3a9caa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:17:59.660049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 576/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Thu Sep 22 12:17:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:01.773126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bb29-f285-425b-88cb-0f6bef5aca8b None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:01.773649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bb29-f285-425b-88cb-0f6bef5aca8b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:01.773649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9bb29-f285-425b-88cb-0f6bef5aca8b None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:01.774987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 591/1167] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:01.780662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b137d-6d56-4674-8866-20b79bc04369 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:01.781071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b137d-6d56-4674-8866-20b79bc04369 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:01.781209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b137d-6d56-4674-8866-20b79bc04369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:01.799330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-789b137d-6d56-4674-8866-20b79bc04369 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:01.800528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-789b137d-6d56-4674-8866-20b79bc04369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:01.865160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 577/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:02.132445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9bb29-f285-425b-88cb-0f6bef5aca8b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:02.229944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9bb29-f285-425b-88cb-0f6bef5aca8b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:02.264461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:02.265048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:02.265324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:02.265592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:02.266243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:02.267603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5148e-872d-4a7a-9398-043300a5ceeb None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:02.282633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 592/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Thu Sep 22 12:18:02 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:18:04.436459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861bbc-60c2-4ee8-99c5-152c9e23bf04 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:04.437021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861bbc-60c2-4ee8-99c5-152c9e23bf04 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:04.437021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19861bbc-60c2-4ee8-99c5-152c9e23bf04 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:04.438338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 578/1170] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:04.443478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:04.443895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:04.444001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:04.462115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:04.463286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:04.479677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:04.537253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 593/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:04.810861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861bbc-60c2-4ee8-99c5-152c9e23bf04 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:04.908266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19861bbc-60c2-4ee8-99c5-152c9e23bf04 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:04.916928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:04.940815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:04.941439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:04.941693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:04.941961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] PATH_INFO: `/v3/domains/3a54a906dfa3470283e1ff15cbb60c05` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:04.952422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3a54a906dfa3470283e1ff15cbb60c05)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:04.953830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e0bc28-56dd-4994-8434-d83f28fbcc70 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:04.955371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 579/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Sep 22 12:18:04 2022] GET /identity/v3/domains/3a54a906dfa3470283e1ff15cbb60c05 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:04.961265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:05.054480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc1011-05a9-48c5-ad28-398f199fca12 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:05.087039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:05.087627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:05.087898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:05.088163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:05.088814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:05.090257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef7c14-ae54-4018-9432-3c30dc1ca778 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:05.115585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 594/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:18:04 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:18:07.272258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9243f6-54dd-471a-8ee0-14899ed9ddb3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:07.272786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9243f6-54dd-471a-8ee0-14899ed9ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:07.272786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9243f6-54dd-471a-8ee0-14899ed9ddb3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:07.274077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 580/1174] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:07.279279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:07.279660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:07.279813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:07.286914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:07.298179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:07.299350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:07.363927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 595/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:07.631064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9243f6-54dd-471a-8ee0-14899ed9ddb3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:07.728931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9243f6-54dd-471a-8ee0-14899ed9ddb3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:07.761141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:07.762318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:07.762594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:07.762864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] PATH_INFO: `/v3/domains/3a54a906dfa3470283e1ff15cbb60c05` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:07.763951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3a54a906dfa3470283e1ff15cbb60c05)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:07.765391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822da658-b9c5-4f1a-868b-b4edfdb80bc9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:07.766906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 581/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Thu Sep 22 12:18:07 2022] GET /identity/v3/domains/3a54a906dfa3470283e1ff15cbb60c05 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:07.772543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:07.865737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd4299-3605-43e0-82fc-23543f90157b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:07.899032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:07.899567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:07.899841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:07.900111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:07.900779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:07.902190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a587e87-a5b6-4a68-9016-4acb6f8ab109 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:07.909988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:07.923678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 596/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:18:07 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:18:10.085629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121a76e-d135-45ff-a523-f3ecc5f123ea None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:10.086173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121a76e-d135-45ff-a523-f3ecc5f123ea None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:10.086173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121a76e-d135-45ff-a523-f3ecc5f123ea None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:10.087500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 582/1178] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:10.093370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:10.093771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:10.093903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:10.112242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:10.113433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:10.177755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 597/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:10.441987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121a76e-d135-45ff-a523-f3ecc5f123ea None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:10.477784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:10.541102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121a76e-d135-45ff-a523-f3ecc5f123ea None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:10.572842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:10.573450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:10.573727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:10.574536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:10.584149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:18:10.588875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:18:10.590531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:18:10.591567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 583/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:18:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:18:10.596448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:10.694752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3114155-d8c3-43fe-b339-d8b06f22b683 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:10.713842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:10.728998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:10.729601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:10.729858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:10.730127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:10.730763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:18:10.731055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:10.732440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:10.740129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 598/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:18:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:10.746621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:10.837690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:10.843821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7740004-abc8-4b4d-9e22-e5a3a97fb29b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:10.877015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:10.877629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:10.877907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:10.878177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] PATH_INFO: `/v3/users/bcac86107f954029bafb9d1084c94ef8` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:10.891366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] RBAC: Authorizing `identity:get_user(user_id=bcac86107f954029bafb9d1084c94ef8)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:10.892840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:10.894407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 584/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:18:10 2022] GET /identity/v3/users/bcac86107f954029bafb9d1084c94ef8 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:10.900109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:10.994101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1843eb-f538-42ae-9419-1a7f12a9c85b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:11.026129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:11.026706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:11.027007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:11.027277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:11.073479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:11.092140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:11.093607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:11.186801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 599/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:18:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:18:11.192631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:11.287299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078c0a8-cc17-4d53-9f49-9ba8cb1e3684 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:11.320446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:11.323063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:11.323063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:11.323063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] PATH_INFO: `/v3/projects/c083cceab00d472e997050c93b14aa46` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:11.323063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] RBAC: Authorizing `identity:get_project(project_id=c083cceab00d472e997050c93b14aa46)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:11.324507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fd508-c066-4d4b-97c6-14b937b7e7aa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:11.325669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 585/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:18:11 2022] GET /identity/v3/projects/c083cceab00d472e997050c93b14aa46 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:11.331809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:11.409148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:11.427215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4355ac9-1d29-48f6-ab24-ce8ef47a8586 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:11.459605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:11.460171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:11.460615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:11.460887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:11.461580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=c083cceab00d472e997050c93b14aa46, user.id=bcac86107f954029bafb9d1084c94ef8) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:18:11.461877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:11.463285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa6059-134c-473f-9d04-8a65d85b2992 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:11.468968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 600/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:18:11 2022] GET /identity/v3/role_assignments?user.id=bcac86107f954029bafb9d1084c94ef8&scope.project.id=c083cceab00d472e997050c93b14aa46&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:13.546840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f5e-0596-4550-8b82-30f2a669409f None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:13.547397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f5e-0596-4550-8b82-30f2a669409f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:13.547397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af82f5e-0596-4550-8b82-30f2a669409f None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:13.548741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 586/1186] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:13.554330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:13.554745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:13.554854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:13.572731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:13.574441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:13.640021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 601/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:13.909431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82f5e-0596-4550-8b82-30f2a669409f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:13.932638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:14.007051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af82f5e-0596-4550-8b82-30f2a669409f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:14.038983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:14.039590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:14.039867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:14.040159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:14.049826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:18:14.054509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:18:14.055629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:18:14.056729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 587/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:18:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:18:14.062218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:14.135791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:14.158519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2a3bf-c35f-4140-a4d3-174bfe18a22a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:14.192284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:14.192896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:14.193174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:14.193443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:14.194090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:18:14.194387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:14.195736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:14.203466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 602/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:18:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:14.209152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:14.286770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:14.304884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822fe41-f59e-4cb6-bd3b-fed57dc2e928 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:14.337945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:14.338551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:14.338824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:14.339089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] PATH_INFO: `/v3/users/bcac86107f954029bafb9d1084c94ef8` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:14.340196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] RBAC: Authorizing `identity:get_user(user_id=bcac86107f954029bafb9d1084c94ef8)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:14.341732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:14.343274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 588/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:18:14 2022] GET /identity/v3/users/bcac86107f954029bafb9d1084c94ef8 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:14.349287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:14.444250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66545ad-d1a1-4ce3-824b-01c9669ec8a2 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:14.475869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:14.476495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:14.476767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:14.477030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] PATH_INFO: `/v3/projects/c083cceab00d472e997050c93b14aa46` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:14.478096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] RBAC: Authorizing `identity:get_project(project_id=c083cceab00d472e997050c93b14aa46)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:14.479501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f27e08-a029-4749-8eb7-e39209d71a3d None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:14.481626 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 603/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:18:14 2022] GET /identity/v3/projects/c083cceab00d472e997050c93b14aa46 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:14.487365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:14.580644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dae41-ab60-40a5-984d-fe2e3f449b88 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:14.613175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:14.613796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:14.614068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:14.614333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] PATH_INFO: `/v3/projects/c083cceab00d472e997050c93b14aa46/users/bcac86107f954029bafb9d1084c94ef8/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:14.616145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] RBAC: Authorizing `identity:create_grant(project_id=c083cceab00d472e997050c93b14aa46, user_id=bcac86107f954029bafb9d1084c94ef8, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:14.617557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b36cd-658c-4dd3-a088-7fb181661a8a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:14.629430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 589/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:18:14 2022] PUT /identity/v3/projects/c083cceab00d472e997050c93b14aa46/users/bcac86107f954029bafb9d1084c94ef8/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:18:16.720752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8242b-75b9-4132-af77-281463a961e6 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:16.721282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8242b-75b9-4132-af77-281463a961e6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:16.721282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8242b-75b9-4132-af77-281463a961e6 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:16.722607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 604/1193] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:18:16.728127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:16.728540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:16.728704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:16.746665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:18:16.747824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:18:16.750470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:16.814938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 590/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:18:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:18:17.078704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8242b-75b9-4132-af77-281463a961e6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:17.087851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:17.176466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8242b-75b9-4132-af77-281463a961e6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:17.208340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:17.208951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:17.209226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:17.209493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:17.220078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:18:17.225145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:18:17.226232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:18:17.227409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 605/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:18:17.232145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:17.324876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0fa5e-ac18-4d11-a0e9-03b0c6246146 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:17.357118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:17.357635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:17.357906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:17.358171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:17.358805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:18:17.359098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:17.360455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:17.367970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 591/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:17.373728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:17.435668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:17.469773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c9fca-946b-46f0-8525-5cfef49d8df0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:17.501635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:17.502244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:17.502520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:17.502790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] PATH_INFO: `/v3/users/bcac86107f954029bafb9d1084c94ef8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:17.503876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] RBAC: Authorizing `identity:get_user(user_id=bcac86107f954029bafb9d1084c94ef8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:17.505325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:17.506853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 606/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/users/bcac86107f954029bafb9d1084c94ef8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:17.513064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:17.515824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:17.609731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cb914-6bb0-4601-a28e-6b01abaa65ae None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:17.641943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:17.642535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:17.642813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:17.643082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:17.706580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:17.708000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:17.709715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:17.801312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 592/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 12:18:17.807033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:17.903288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bf1ae-05c0-44f0-8990-1a5a2a837848 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:17.935748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:17.936383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:17.936658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:17.936928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] PATH_INFO: `/v3/projects/c083cceab00d472e997050c93b14aa46` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:17.938016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] RBAC: Authorizing `identity:get_project(project_id=c083cceab00d472e997050c93b14aa46)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:17.939431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a88f6e-f4be-41a0-a7db-590eba23b1eb None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:17.941051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 607/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/projects/c083cceab00d472e997050c93b14aa46 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:17.947042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:18:18.041335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc067c8-8abf-4735-887f-79a46c2c6309 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:18:18.043282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:18.075650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:18:18.076231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:18:18.076580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:18:18.076847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:18:18.077549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=c083cceab00d472e997050c93b14aa46, user.id=bcac86107f954029bafb9d1084c94ef8) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:18:18.077852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:18:18.079256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc34e524-2603-40c7-89bd-e5b9112e08d9 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:18:18.084774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 593/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:18:17 2022] GET /identity/v3/role_assignments?user.id=bcac86107f954029bafb9d1084c94ef8&scope.project.id=c083cceab00d472e997050c93b14aa46&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:18:27.931078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:18:28.082281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=17 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:13.231609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a6ff-c402-45f3-bcd7-fb484915a009 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:13.232161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a6ff-c402-45f3-bcd7-fb484915a009 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:13.232161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20a6ff-c402-45f3-bcd7-fb484915a009 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:13.233623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 608/1201] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:13.238880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:13.239375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:13.239476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:13.258495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:13.260234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:13.338816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 594/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:13.611777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20a6ff-c402-45f3-bcd7-fb484915a009 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:13.729149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20a6ff-c402-45f3-bcd7-fb484915a009 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:13.767978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:13.768617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:13.768893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:13.769158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:13.782501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:13.783952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:13.798901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:13.804655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:13.805755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:13.805755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:13.805755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:13.805755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 609/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:13.808094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:13.951662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d93f-7f83-4982-9080-39438bc5269b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:13.988715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:13.998966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:13.999605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:14.000086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:14.000627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:14.001573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:14.002064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:14.003783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a622fb7-a901-4e38-8e20-12d86e023d90 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:14.020543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 595/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:14.027143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:14.105286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:14.173165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7e69b-ed53-43f3-85ae-e4d099e6f2c1 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:14.216038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:14.216684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:14.216959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:14.217225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:14.217903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:14.219316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ad253-4969-4b15-8ca8-3f20b964f23e None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:14.250118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 610/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Thu Sep 22 12:19:14 2022] POST /identity/v3/users => generated 251 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:19:16.677033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150c72-fdf9-4586-947e-b5946516507e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:16.677574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150c72-fdf9-4586-947e-b5946516507e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:16.677574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4150c72-fdf9-4586-947e-b5946516507e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:16.679205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 596/1206] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:16.685790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:16.686262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:16.686377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:16.707502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:16.708956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:16.791077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 611/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:17.109266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4150c72-fdf9-4586-947e-b5946516507e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:17.237277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4150c72-fdf9-4586-947e-b5946516507e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:17.285528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:17.286172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:17.286479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:17.286798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:17.300317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:17.305100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:17.306106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:17.308209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 597/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:19:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:17.312851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:17.362077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:17.444677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a84d68-47a3-4c13-9c81-497552954c58 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:17.496122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:17.496992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:17.497677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:17.497956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:17.498653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:17.498937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:17.500327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:17.515000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 612/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:19:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:19:17.516303 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:17.585012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:17.650266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea71dc-f75c-4540-92cd-62ede6236104 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:17.691661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:17.692356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:17.692627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:17.692827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:17.704855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:17.706336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:17.728835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:17.730192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:17.731288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:17.731288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:17.731288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:17.736673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 598/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:17.736835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:17.840389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:17.843907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc26d2d-ecea-4404-aac7-524b1f44fd1c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:17.895557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:17.896208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:17.896600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:17.896776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:17.897433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:17.897762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:17.899758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:17.912926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 613/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:17.918368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:18.025934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37937b-6d37-4bdd-be30-78f89757da0e None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:18.028150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:18.063698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:18.064349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:18.064630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:18.064898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:18.072979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:18.074403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:18.085587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:18.087626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:18.088984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:18.092064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 599/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:19:18.093149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:18.212643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cadcb-9332-4285-a54e-bda6c08af13a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:18.249733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:18.250373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:18.250645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:18.250911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:18.251669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:18.251971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:18.253550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:18.276359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 614/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:18.282044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:18.417607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369da3-e537-4611-9be2-5da403a84f90 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:18.457664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:18.458394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:18.458745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:18.459086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:18.499065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:18.553936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:18.555722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:18.674324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 600/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:19:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 12:19:18.679942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:18.703356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:18.815715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b774cfd9-0952-4f38-a7d2-5d5101623104 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:18.876267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:18.876955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:18.877232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:18.877502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:18.890209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:18.892248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:18.909191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:18.910513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:18.911534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:18.911534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:18.911534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:18.911534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 615/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:19:18.917586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:18.990855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:19.046345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef171e11-11bf-4ac3-a05f-06cb27309d17 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:19.094754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:19.095385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:19.095686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:19.095954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:19.096699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:19.097030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:19.098738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:19.110367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 601/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:19:19.119703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:19.201909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:19.242460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f529d-6c73-4775-82fc-8880f1625927 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:19.280671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:19.281297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:19.281582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:19.281847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:19.289343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:19.291175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:19.304630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:19.306453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:19.307375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 616/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:19.312279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:19.425639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866a64f-7575-4327-b306-b3d641d56f26 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:19.451191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:19.464374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:19.465015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:19.465288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:19.465601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:19.466409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:19.466713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:19.468174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaba971c-fa88-40ce-81f0-ffd7c1eaf50c None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:19.480860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 602/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:19.486991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:19.602293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c89d70-0d14-44f1-b549-a34ab8e8988c None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:19.632517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:19.641327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:19.642010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:19.642284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:19.642549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:19.643258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=faf0bcc4ab8649cf98889340287a227c) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:19.643563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:19.645044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02499873-9686-4115-a589-e4d5d3d6b3cb None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:19.652826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 617/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:19:19 2022] GET /identity/v3/role_assignments?user.id=faf0bcc4ab8649cf98889340287a227c&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:21.812041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453102-4bd9-4757-9a57-4fa7a7b82545 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:21.812621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453102-4bd9-4757-9a57-4fa7a7b82545 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:21.812621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94453102-4bd9-4757-9a57-4fa7a7b82545 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:21.814119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 603/1220] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:19:21.819506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:21.819987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:21.820097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:21.839410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:21.840704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:21.913204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 618/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:22.183794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94453102-4bd9-4757-9a57-4fa7a7b82545 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:22.297638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94453102-4bd9-4757-9a57-4fa7a7b82545 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:22.331448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:22.332049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:22.332355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:22.332629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:22.343686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:22.348714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:22.349937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:22.350784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 604/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:19:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:22.356492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:22.458444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a972b-bc0f-4b29-9a54-9909b0aeb060 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:22.483285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:22.494323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:22.495158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:22.495424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:22.495698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:22.496392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] RBAC: Adding query filter params (name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:22.496737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:22.498068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:22.506183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 619/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:19:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:22.512723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:22.562445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:22.626237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf083c-d0b1-41ab-8540-3cbc310b94e6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:22.663950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:22.664800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:22.665071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:22.665346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:22.675486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:22.676982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:22.691299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:22.692517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:22.693717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:22.693717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:22.693717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:22.697816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 605/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:19:22.698759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:22.816669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5e9d1-85d8-4f56-9965-eb5ab560d413 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:22.854751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:22.855383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:22.855614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:22.855878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:22.856776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:22.857137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:22.858922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:22.881283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 620/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:22.882553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.018692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:23.026546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0a92fa-218e-4383-9f7c-f21b4095b96b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.069538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.070189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.070459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.071303 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.078796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.080227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:23.091264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:23.092238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:23.098589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 606/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:23.099333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.157517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:23.217260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65e1a3-4dba-465c-ad77-be10d84dbd57 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.252878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.253489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.253760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.254026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.254726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:23.255022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.256460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.273545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 621/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:23.279837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.391885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2664c87-0120-41dd-8cf2-046f979ad1bd None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.429217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.430027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.430793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.431063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.443765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.445260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:23.460317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:23.462002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:23.463064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:23.463064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:23.463064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:23.463064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 607/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:23.467667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.568903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131aa457-7b0a-4649-9a45-2da053883d76 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.571454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:23.606093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.606692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.606963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.607226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.607840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:23.608133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.609513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.621757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 622/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:23.628258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.638092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:23.740794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f3580-3537-4a4a-a36b-001afa3fd5e6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.776694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.777310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.777587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.777850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.784353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.785770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:23.797278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:23.798454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:23.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 608/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:23.805493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:23.915394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06f8fb-4a20-4083-a596-ef283c0373ea None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:23.950810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:23.951419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:23.951691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:23.951958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:23.952730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:23.953106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:23.954661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459c168-477f-4fa2-b27c-76c87b1971ae None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:23.967668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 623/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:23.975574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:24.042119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:24.078613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91028c09-3c13-4a88-9e37-fa41a20ce7c1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:24.113669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b40d62-808e-476c-aa65-532369201426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:24.114318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40d62-808e-476c-aa65-532369201426 None admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:24.114594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40d62-808e-476c-aa65-532369201426 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:24.114870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b40d62-808e-476c-aa65-532369201426 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/faf0bcc4ab8649cf98889340287a227c/roles/dd3b9bd9959d40ee985145b693e8b53b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:24.128632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b40d62-808e-476c-aa65-532369201426 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=faf0bcc4ab8649cf98889340287a227c, role_id=dd3b9bd9959d40ee985145b693e8b53b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:24.130050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b40d62-808e-476c-aa65-532369201426 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:24.144730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 609/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:19:23 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/faf0bcc4ab8649cf98889340287a227c/roles/dd3b9bd9959d40ee985145b693e8b53b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:19:26.324898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb50b-3cd2-41cb-addc-774e6b712be8 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:26.325410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb50b-3cd2-41cb-addc-774e6b712be8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:26.325501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb50b-3cd2-41cb-addc-774e6b712be8 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:26.326776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 624/1233] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:26.332047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:26.332486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:26.332776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:26.353474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:26.354834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:26.425768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 610/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 12:19:26.700859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb50b-3cd2-41cb-addc-774e6b712be8 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:26.730400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:26.814301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb50b-3cd2-41cb-addc-774e6b712be8 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:26.850437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:26.851132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:26.851342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:26.851604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:26.864249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:26.869090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:26.871072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:26.873034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 625/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Thu Sep 22 12:19:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:26.880939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:26.928242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:26.987076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0b596-b5d8-45af-8d62-d3b75d45d1bd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.029449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.030257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.030550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.030948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:27.031725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] RBAC: Adding query filter params (name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:27.032222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:27.034141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:27.044002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 611/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Thu Sep 22 12:19:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:27.050079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:27.134979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:27.155509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22a08ca-1db9-4b88-a0de-883a8e614a0b None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.200786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.201718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.202115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.202540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:27.215866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:27.217906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:27.236981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:27.238249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:27.239206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:27.239206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:27.239206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:27.239206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 626/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:27.245228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:27.365065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba47b8-1077-429f-8578-fbb8d0d20055 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.375687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:27.404624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.405232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.405508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.405779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:27.406416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:27.406717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:27.408081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:27.421566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 612/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:27.427412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:27.546956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af26d5-422a-4f80-8654-60c56cf34ada None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.590138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.590762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.591067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.591306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:27.594258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:27.600751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:27.603325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:27.622036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:27.623198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:27.624428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 627/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:27.629400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:27.745520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db2033-eda6-49df-893b-ac982b53d609 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.781825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.782423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.782701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.782975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:27.783693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:27.783997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:27.785447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:27.803053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 613/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:27.809363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:27.921030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88daf1-cdc2-4783-aa2b-355ab1b42da4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:27.960638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:27.962025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:27.962301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:27.962566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:28.036337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:28.048274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:28.049768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:28.155463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 628/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:19:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:19:28.161367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:28.190595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:28.265282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d615bb9-78df-4f13-8db7-fc99e6773a36 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:28.299360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:28.299981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:28.300259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:28.300560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:28.311067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:28.312518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:28.328080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:28.329139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:28.330049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:28.330049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:28.330049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:28.330049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 614/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:28.337215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:28.437359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:28.445917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0bbb92-48eb-4266-b34b-5a133ed966c9 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:28.486049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:28.486700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:28.486986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:28.487239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:28.487864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:28.488161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:28.489591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:28.506216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 629/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:19:28.507417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:28.605708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:28.632017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6df5-22f0-48d9-a491-133dd9f09557 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:28.668710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:28.669425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:28.669766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:28.670039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:28.678416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:28.679902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:28.695048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:28.696163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:28.697317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 615/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 12:19:28.702367 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:28.825312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea10572-93a6-4f8b-b50d-d35026124484 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:28.861754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:28.862464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:28.862707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:28.862966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:28.863672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:28.863973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:28.865476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723c17e-4f4e-4948-8461-ce54ba96bcca None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:28.878431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 630/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:28.884555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:28.997645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb2f053-fa66-4b58-957c-e92a30c28ad1 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:29.032119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:29.032909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:29.033221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:29.033605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:29.034390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] RBAC: Adding query filter params (role.id=dd3b9bd9959d40ee985145b693e8b53b, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=faf0bcc4ab8649cf98889340287a227c) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:29.034695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:29.036227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad3e6d-1957-4dce-b3c5-80804922b2c1 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:29.043286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 616/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:19:28 2022] GET /identity/v3/role_assignments?user.id=faf0bcc4ab8649cf98889340287a227c&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=dd3b9bd9959d40ee985145b693e8b53b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:31.390115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431f59-544d-48a6-9b14-49257f50685d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:31.390738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431f59-544d-48a6-9b14-49257f50685d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:31.390738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c431f59-544d-48a6-9b14-49257f50685d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:31.391994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 631/1247] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:31.397511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:31.398006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:31.398116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:31.405669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:31.420068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:31.421555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:31.495031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 617/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:31.806154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c431f59-544d-48a6-9b14-49257f50685d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:31.825305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:31.923194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c431f59-544d-48a6-9b14-49257f50685d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:31.964680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:31.965305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:31.965604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:31.965856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:31.978019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:31.983564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:31.984831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:31.986070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:31.990434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 632/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:19:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:31.991583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.104759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b513054b-ac13-40a9-aac7-dfc099dc2299 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:32.136469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:32.143412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:32.144812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:32.145154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:32.145487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:32.146333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:32.146717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:32.148080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:32.158195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 618/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:19:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:32.164582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.253370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:32.277947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3057d-b694-4f49-bb0c-c956fa3e0328 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:32.316227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:32.316911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:32.317177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:32.317433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:32.328440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:32.329919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:32.346922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:32.348259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:32.349214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:32.349214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:32.349214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:32.349214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 633/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:32.354964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.457693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a3682-a37e-4cfd-b568-3c9fb4585474 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:32.492856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:32.493465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:32.493740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:32.494011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:32.494647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:32.494949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:32.496346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:32.513735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 619/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:32.514922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.629761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab11622-3927-4930-a5db-be728627b2b7 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:32.657063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:32.665963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:32.666609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:32.666881 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:32.667192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] PATH_INFO: `/v3/users/placement` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:32.675585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:32.677092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:32.689142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:32.690354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:32.691496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 634/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:32.696181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.752796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:32.797218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797c72e-7183-4033-8310-c29c81301b2b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:32.830187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:32.830787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:32.831071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:32.831341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:32.832056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:32.832421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:32.833850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:32.853442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 620/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:32.859330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:32.975278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39aee7f-79b9-4919-b146-d44d3d345bca None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.011609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.012218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.012533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.012805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.065673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:33.078381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.079813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.173467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 635/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:19:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 12:19:33.179396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:33.277692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0578ae4b-47f3-45d7-8756-a2c265c1a7f8 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.290026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:33.310547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.311155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.311428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.311698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.321811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.323248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:33.337134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:33.342902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:33.343977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:33.343977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:33.343977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:33.343977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 621/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:33.344214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:33.427471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:33.440013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e89c0a-2675-4c65-9134-589ff71f992a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.473436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.474023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.474300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.474570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.475202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:33.475502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.476924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.487646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 636/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:33.493643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:33.596525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a646e26-e61c-4c52-bca7-b6c34b832862 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.628681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.629289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.629563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.629826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.636600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.638069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:33.649363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:33.650275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:33.651256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 622/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:33.656279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:33.750698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbad966-188a-429a-b81b-4f3f6aba1a0a None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.782561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.783138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.783409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.783673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.784359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:33.784655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.786052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb26ee-b526-455a-90b8-0d835ff48c13 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.797300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 637/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:33.803236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:33.828349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:33.901058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00808-d969-4d77-a367-c494ebf7467a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:33.932820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:33.933404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:33.933674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:33.933941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:33.934641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=faf0bcc4ab8649cf98889340287a227c) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:33.934941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:33.936377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33c86c-445c-443a-9d7a-8ce7df772dca None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:33.941496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 623/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:19:33 2022] GET /identity/v3/role_assignments?user.id=faf0bcc4ab8649cf98889340287a227c&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:36.039417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf330-623c-41ab-9bf7-afe65e606f28 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:36.039973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf330-623c-41ab-9bf7-afe65e606f28 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:36.039973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf330-623c-41ab-9bf7-afe65e606f28 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:36.041272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 638/1261] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:36.046273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:36.046642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:36.046842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:36.064522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:36.065691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:36.131569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 624/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:36.397552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf330-623c-41ab-9bf7-afe65e606f28 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:36.408055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:36.494029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf330-623c-41ab-9bf7-afe65e606f28 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:36.525962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:36.526561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:36.526831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:36.527094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:36.536583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:36.541532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:36.542561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:36.543418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 639/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:19:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:36.549256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:36.575232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:36.661398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac114f0c-0c8f-4d75-914d-d467e7937882 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:36.694669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:36.695291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:36.695568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:36.695837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:36.696504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:36.696805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:36.698152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:36.705788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 625/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:19:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:36.711517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:36.775026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:36.807380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842115ba-7613-4620-9c9f-28f090abe259 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:36.838638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:36.839238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:36.839510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:36.839774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:36.849408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:36.850817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:36.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:36.866275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:36.868119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:36.868119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:36.868119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:36.868119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 640/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:36.872520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:36.949221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:36.968094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f0c25-e44f-4a4d-8b5b-ebb557bf79fe None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.002023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.002614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.002885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.003148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.003761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:37.004055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.005434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.017873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 626/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:37.024351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.120047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327f1095-be8a-42c4-b62f-731ca52ce4f4 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.138611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:37.153064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.153678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.153955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.154222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.161345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.162772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:37.180075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:37.181195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:37.182348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 641/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:37.187001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.281617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3bc00-8b6f-420c-867f-7815d84163fa None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.313427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.314005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.314276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.314539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.315229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:37.315525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.316944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.325090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:37.333141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 627/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:37.339323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.435113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395bae1-ef6b-4ecc-b5d8-9188be7a9ec5 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.467389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.467985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.468253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.468540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.478735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.480141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:37.493587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:37.494608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:37.495618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:37.495618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:37.495618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:37.495618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 642/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:37.500172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.599700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73572c-9915-462c-a375-3543f36c2ca5 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.633741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.634324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.634596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.634864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.635478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:37.635771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.637153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.647786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 628/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:37.654062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.684416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:37.751190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dc8f4-739b-4f9d-adf5-b42acc6eba89 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.783396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.783998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.784271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.784569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.790946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.792409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:37.803273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:37.804325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:37.805919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 643/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:37.810478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:37.866074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:37.908547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a170d3-dfd8-4006-ae8e-918b266b2580 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:37.939995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:37.940606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:37.940879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:37.941145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:37.941807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:37.942103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:37.943511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b54ca5-5ec9-4818-93df-d84d9d0d4881 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:37.955325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 629/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:37.961310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:38.048325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:38.057643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72648cb-914f-48d3-b0d6-4582c8d0221f None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:38.089215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:38.089846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:38.090116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:38.090378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] PATH_INFO: `/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/faf0bcc4ab8649cf98889340287a227c/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:38.092185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] RBAC: Authorizing `identity:create_grant(project_id=6084e6bd20cd4e1fa82581fec46a2f78, user_id=faf0bcc4ab8649cf98889340287a227c, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:38.093622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28150840-a80f-439b-b4f3-1eae8744ff98 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:38.106444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 644/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Thu Sep 22 12:19:37 2022] PUT /identity/v3/projects/6084e6bd20cd4e1fa82581fec46a2f78/users/faf0bcc4ab8649cf98889340287a227c/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:19:40.225634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305749f-1a23-4f2a-a0ae-1d1b1e3812ce None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:40.226130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305749f-1a23-4f2a-a0ae-1d1b1e3812ce None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:40.226130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305749f-1a23-4f2a-a0ae-1d1b1e3812ce None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:40.227447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 630/1274] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:40.232870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:40.233235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:40.233409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:40.251450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:40.252644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:40.317138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 645/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:40.583969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305749f-1a23-4f2a-a0ae-1d1b1e3812ce None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:40.686328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305749f-1a23-4f2a-a0ae-1d1b1e3812ce None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:40.702342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:40.718653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:40.719256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:40.719529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:40.719795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:40.730118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 12:19:40.734791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:40.735881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:40.736827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 631/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Thu Sep 22 12:19:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:40.741899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:40.818486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:40.837172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31c6e5-dff5-4d11-bd1c-62152e090eef None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:40.869727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:40.870323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:40.870595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:40.870858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:40.871502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:40.871794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:40.873174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:40.880983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 646/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Thu Sep 22 12:19:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:40.886653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:40.982821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a0ef6-2949-417a-8b1d-1d58cca067e4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.016661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.017270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.017547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.017817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.028077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.030167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:41.045119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:41.046296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:41.047489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:41.047489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:41.047489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:41.047489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 632/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:41.051816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:41.147172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956c415a-3f26-4eed-a4cb-efcb64ae7bec None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.178587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.179161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.179431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.179694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.180353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:41.180649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.181980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.192948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 647/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:41.198909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:41.228766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:41.295057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53109bcd-a6d7-4319-b63b-f06b96d1497a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.327434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.328036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.328332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.328608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.335634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.337649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 12:19:41.348557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:41.349646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:41.350628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 633/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:41.355336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:41.358975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:41.449887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a32cc-c83d-44b7-b4ec-37eabdd84d60 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.482284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.482857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.483129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.483395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.484090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:41.484414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.485812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.502438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 648/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:41.508039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:41.597204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:41.607175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3b4170-8b4c-4931-b48e-f85dba52d4e3 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.640280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.640892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.641170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.641441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.705332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.706694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.789469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:41.799501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 634/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 12:19:41.805795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:41.866480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:41.904096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8d553-9329-44c4-858e-5f2eed80b3ef None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:41.936809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:41.937413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:41.937680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:41.937939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:41.947327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:41.948769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 12:19:41.962586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:41.963658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 12:19:41.964924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 12:19:41.964924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 12:19:41.964924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:41.964924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 649/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:41.969274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:42.063512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a2c2f-d746-4a79-b2b7-8412b4c4f733 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:42.096534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:42.097127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:42.097405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:42.097927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:42.098552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:42.098852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:42.100214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:42.111623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 635/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Thu Sep 22 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:42.117471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:42.213739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae64d74c-ef77-4d54-9cb2-1dc5a4638ff0 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:42.226688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:42.246628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:42.247237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:42.247511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:42.247779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:42.254108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:42.255525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 12:19:42.266970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:42.268026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: ERROR keystone.server.flask.application Sep 22 12:19:42.269175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 650/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Thu Sep 22 12:19:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:42.273925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:42.325445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:42.382229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da05690e-6ab5-47c5-9762-87c0adab06d4 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:42.414607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:42.415200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:42.415476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:42.415747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:42.416451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:42.416754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:42.418164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc960b15-1680-402f-acab-311d883f4e6b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:42.429650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 636/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Thu Sep 22 12:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:42.436373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:42.531491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcbab6-6ef2-46cc-a19b-8b1002ad4a79 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:42.563919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:42.564525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:42.564797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:42.565063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:42.565760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] RBAC: Adding query filter params (role.id=97ed8ec57a984d71b7ba263e643b38fe, scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78, user.id=faf0bcc4ab8649cf98889340287a227c) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:42.566060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:42.567473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbee82-14c6-4aa7-9767-773d9e413b75 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:42.573541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 651/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Thu Sep 22 12:19:42 2022] GET /identity/v3/role_assignments?user.id=faf0bcc4ab8649cf98889340287a227c&scope.project.id=6084e6bd20cd4e1fa82581fec46a2f78&role.id=97ed8ec57a984d71b7ba263e643b38fe => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:44.716425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d73843-4dae-4a3a-a057-8109ce0a7323 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:44.716963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d73843-4dae-4a3a-a057-8109ce0a7323 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:44.716963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d73843-4dae-4a3a-a057-8109ce0a7323 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:44.718280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 637/1288] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:44.723769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:44.724167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:44.724337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:44.742483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:44.743652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:44.771305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:44.807456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 652/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:45.073848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d73843-4dae-4a3a-a057-8109ce0a7323 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:45.169305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d73843-4dae-4a3a-a057-8109ce0a7323 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:45.179250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:45.202082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:45.202682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:45.202951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:45.203215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:45.203770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:45.205149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:45.214054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:45.215263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:45.216171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 638/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:19:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:45.221025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:45.314628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e5db7-0337-474f-893f-058f2f238784 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:45.354293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:45.354819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:45.355093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:45.355362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:45.355977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:45.356273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:45.357660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a31a5-07b6-48ae-921b-50cff40e6383 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:45.362417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 653/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 22 12:19:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:45.368707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:45.462831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faf45b-6df8-4873-a048-82c54d141ef6 None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:45.494893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:45.495472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:45.495746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:45.496010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] PATH_INFO: `/v3/services` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:45.496637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:45.496931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:45.498285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73866cb-3f33-4755-bdc8-2abbeb0b3e46 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:45.502456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 639/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 22 12:19:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:47.638792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f3c0-d37d-454d-be72-57fa90c7cd3c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:47.639351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f3c0-d37d-454d-be72-57fa90c7cd3c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:47.639516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8f3c0-d37d-454d-be72-57fa90c7cd3c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:47.641637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 654/1293] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:47.647658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f19b0-fd94-411d-91b8-1f03ba7622ff None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:47.648105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f19b0-fd94-411d-91b8-1f03ba7622ff None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:47.648198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f19b0-fd94-411d-91b8-1f03ba7622ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:47.668647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a25f19b0-fd94-411d-91b8-1f03ba7622ff None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:47.669940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a25f19b0-fd94-411d-91b8-1f03ba7622ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:47.701637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:47.739005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 640/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:48.009781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f3c0-d37d-454d-be72-57fa90c7cd3c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:48.039697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:48.106991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8f3c0-d37d-454d-be72-57fa90c7cd3c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:48.139256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:48.139852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:48.140126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:48.140428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:48.141061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:48.142411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8491b4-2318-4c1d-8ce4-dbdb2ac83b8c None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:48.156508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 655/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Thu Sep 22 12:19:48 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:19:50.283169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9596-dadd-4050-9bcd-746fcb2ee4d7 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:50.283667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9596-dadd-4050-9bcd-746fcb2ee4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:50.283667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf9596-dadd-4050-9bcd-746fcb2ee4d7 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:50.284981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 641/1296] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:50.290759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:50.291154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:50.291290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:50.309400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:50.310568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:50.379804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 656/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:50.646248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9596-dadd-4050-9bcd-746fcb2ee4d7 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:50.743448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf9596-dadd-4050-9bcd-746fcb2ee4d7 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:50.775350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:50.775948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:50.776223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:50.776515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:50.777078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:50.778422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:50.786775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:50.787927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:50.789146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 642/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:19:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 142 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:50.794258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:50.811768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:50.890932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e3f5eb-8007-43e5-a473-d95419575184 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:50.923108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:50.923646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:50.923921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:50.924190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:50.924831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:50.925127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:50.926481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:50.931434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 657/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 22 12:19:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:50.937359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:50.968334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:51.034610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e7882-ac2d-478c-a387-4f0212e5db1a None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:51.065970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:51.066555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:51.066828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:51.067092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:51.067765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] RBAC: Adding query filter params (interface=public, service_id=e2d0a554b4bf4c68be459c0c95e82ac3, region_id=RegionOne) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:51.068062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:51.069454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae27a2-31bc-4f8b-8517-bce0f41ba18e None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:51.074769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 643/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Thu Sep 22 12:19:50 2022] GET /identity/v3/endpoints?service_id=e2d0a554b4bf4c68be459c0c95e82ac3&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:51.080377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:51.159070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:51.177669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169850-3ecb-4a2b-830e-2fb9ef56dd04 None admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:51.210443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:51.211026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:51.211837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:51.212103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:51.212713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:51.214089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010d1dc-1b20-4f3e-bf0e-32e38b18f9be None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:51.220597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 658/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Thu Sep 22 12:19:51 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:19:53.323436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14525a0-269d-4d63-aae1-418b94a1b7a6 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:53.323947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14525a0-269d-4d63-aae1-418b94a1b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:53.323947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14525a0-269d-4d63-aae1-418b94a1b7a6 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:53.325325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 644/1302] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:19:53.330524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:53.330934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:53.331054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:53.348900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:19:53.350066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:19:53.413040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 659/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:19:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 12:19:53.679055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14525a0-269d-4d63-aae1-418b94a1b7a6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:53.769517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:53.776828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14525a0-269d-4d63-aae1-418b94a1b7a6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:53.808987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:53.809591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:53.809865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:53.810133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:53.810692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:53.812043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 12:19:53.823184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 12:19:53.824355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application with Lock( Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self._enter() Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application return self.creator() Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application created_value = creator( Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: ERROR keystone.server.flask.application Sep 22 12:19:53.825344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 645/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Thu Sep 22 12:19:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 12:19:53.830870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:53.905036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:19:53.927287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d713970-b1a5-4a88-a41b-f5e2063ac672 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:53.959543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:53.960077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:53.960370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:53.960639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] PATH_INFO: `/v3/services` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:53.961842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:19:53.962134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:53.963471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63225fc0-bbf4-4420-9ec2-0f171b7f14e4 None admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:53.968332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 660/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Thu Sep 22 12:19:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:19:53.974367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:19:54.068677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138151bc-7ac5-4621-bbce-db93afdbd48b None admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:19:54.101220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:19:54.101818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:19:54.102095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:19:54.102363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:19:54.103001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:19:54.104355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e5a82-1571-452d-845c-d8066684fe11 None admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:19:54.122589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 646/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:19:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:20:03.965438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=30 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:20:04.118407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:20:58.248264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df203-c6e9-4d18-b656-cd83a09ef026 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:20:58.248740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df203-c6e9-4d18-b656-cd83a09ef026 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:20:58.248957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273df203-c6e9-4d18-b656-cd83a09ef026 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:20:58.257691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbffc50-3826-46c1-bb08-c8c06e1533ca None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:20:58.258110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbffc50-3826-46c1-bb08-c8c06e1533ca None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:20:58.258261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbffc50-3826-46c1-bb08-c8c06e1533ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:20:58.263026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 661/1307] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:20:58 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:20:58.406283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-acbffc50-3826-46c1-bb08-c8c06e1533ca None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:20:58.407615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-acbffc50-3826-46c1-bb08-c8c06e1533ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:20:58.523250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 647/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:20:58.817203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:20:58.817558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:20:58.817744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:20:58.819244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 662/1309] 10.222.0.33 () {54 vars in 964 bytes} [Thu Sep 22 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:20:58.825493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646f81-9ec1-4ca3-816e-b9de51c6ad03 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:20:58.825855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646f81-9ec1-4ca3-816e-b9de51c6ad03 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:20:58.826199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f646f81-9ec1-4ca3-816e-b9de51c6ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:20:58.927668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2f646f81-9ec1-4ca3-816e-b9de51c6ad03 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:20:58.929039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f646f81-9ec1-4ca3-816e-b9de51c6ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:20:58.950017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:20:59.009165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 648/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Sep 22 12:20:58 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:20:59.018046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:20:59.019821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:20:59.138812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b515f45-c56e-4c5d-9c89-67dc34462638 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:20:59.214349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:20:59.214950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:20:59.215218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:20:59.215473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:20:59.217078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:20:59.273537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:20:59.329954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:20:59.331703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ae4e3-a4d8-45d1-a425-c6c33c69475e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:20:59.353577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 663/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:01.786698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f101f-cab5-478c-8e5b-bc00b904677a None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:01.787211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f101f-cab5-478c-8e5b-bc00b904677a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:01.787211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f101f-cab5-478c-8e5b-bc00b904677a None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:01.789278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 649/1312] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:01.794193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05a2e-3586-4c5d-87f8-b4958323672a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:01.794585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05a2e-3586-4c5d-87f8-b4958323672a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:01.794750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce05a2e-3586-4c5d-87f8-b4958323672a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:01.864142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7ce05a2e-3586-4c5d-87f8-b4958323672a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:01.865397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7ce05a2e-3586-4c5d-87f8-b4958323672a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:01.963608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 664/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:02.263999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:02.264454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:02.264730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:02.266079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 650/1314] 10.222.0.33 () {54 vars in 964 bytes} [Thu Sep 22 12:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:02.272663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c062-d9f5-412f-8255-3ad00dab5286 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:02.272970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c062-d9f5-412f-8255-3ad00dab5286 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:02.273237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4c062-d9f5-412f-8255-3ad00dab5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:02.284988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:02.353156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5fc4c062-d9f5-412f-8255-3ad00dab5286 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:02.354351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5fc4c062-d9f5-412f-8255-3ad00dab5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:02.430669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 665/1315] 10.222.0.33 () {58 vars in 1061 bytes} [Thu Sep 22 12:21:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:02.439645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:02.441241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:02.543898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546a7f-ae0f-4f20-889c-c89d9de4d99d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:02.616555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:02.618572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:02.619152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:02.619448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:02.619903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:02.621870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:02.729598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:02.731073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac974763-741d-4f79-b25e-c031f013a68d service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:02.748815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 651/1316] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:05.414354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:05.414890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:05.414890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:05.416230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 666/1317] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:05.421570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52862e3-612f-4ddd-a01c-9ab28cd0602b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:05.422000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52862e3-612f-4ddd-a01c-9ab28cd0602b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:05.422106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52862e3-612f-4ddd-a01c-9ab28cd0602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:05.486355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d52862e3-612f-4ddd-a01c-9ab28cd0602b None None] MFA Rules not processed for user `6c34613c69674b59957862a501334be0`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:05.487814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d52862e3-612f-4ddd-a01c-9ab28cd0602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:05.538270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:05.581741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 652/1318] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:05 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:05.868368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:05.869905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:05.970160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cbd72d-8a0e-40f1-867b-0111de7f1d8d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:05.973200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:06.043110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:06.043702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:06.043974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:06.044240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:06.045836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:06.150030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:06.151374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad0639-49f5-46a4-869b-2fad28272e99 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:06.168505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 667/1319] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:05 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:08.964985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:08.965534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:08.965534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:08.966894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 653/1320] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:08.972567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bea7eb-d49d-4828-bfc0-2900be615778 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:08.972937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bea7eb-d49d-4828-bfc0-2900be615778 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:08.973100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bea7eb-d49d-4828-bfc0-2900be615778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:09.013080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:09.038184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d0bea7eb-d49d-4828-bfc0-2900be615778 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:09.039373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d0bea7eb-d49d-4828-bfc0-2900be615778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:09.131971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 668/1321] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:09.417739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:09.419267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:09.523275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe83f43-2cab-4672-b3fc-7a9459f3f888 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:09.597450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:09.598036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:09.598307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:09.598571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:09.600136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:09.607010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:09.704858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:09.706203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf9b1d-e664-4051-a3ea-8b31e4b18c91 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:09.724832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 654/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:09 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:12.506616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:12.507197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:12.507197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:12.508502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 669/1323] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:12.513690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243a46-6218-4fb5-9bc1-fcc43dad84ab None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:12.514091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243a46-6218-4fb5-9bc1-fcc43dad84ab None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:12.514238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2243a46-6218-4fb5-9bc1-fcc43dad84ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:12.588013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a2243a46-6218-4fb5-9bc1-fcc43dad84ab None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:12.589277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a2243a46-6218-4fb5-9bc1-fcc43dad84ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:12.649851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:12.692409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 655/1324] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:12.972498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:12.974047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:13.076410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935df336-c33f-4ae9-999d-8b612896eb6a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:13.089749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:13.153060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:13.153579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:13.153850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:13.154113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:13.155762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:13.263235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:13.264706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805435cd-1d0a-4e19-9791-869d34b08c9e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:13.284916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 670/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:15.485341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:15.485873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:15.485873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:15.487361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 656/1326] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:21:15.492954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768a7f-3fc1-4df2-bbbb-f4e798c7a6f5 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:15.493343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768a7f-3fc1-4df2-bbbb-f4e798c7a6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:15.493891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768a7f-3fc1-4df2-bbbb-f4e798c7a6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:15.541491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:15.562441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3a768a7f-3fc1-4df2-bbbb-f4e798c7a6f5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:15.563862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3a768a7f-3fc1-4df2-bbbb-f4e798c7a6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:15.668213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 671/1327] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:15.945541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:15.947059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:16.049541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd899115-42aa-40c0-ac21-72ea26394c2d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:16.123884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:16.124445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:16.124722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:16.124988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:16.126578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:16.144800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:16.241500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:16.242937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d4f83-629a-43b5-9e11-50abb77f8bf2 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:16.261585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 657/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:18.477720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:18.478206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:18.478206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:18.479518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 672/1329] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:18.484836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d006f-a023-4857-a6dd-7c0f94c01769 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:18.485235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d006f-a023-4857-a6dd-7c0f94c01769 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:18.485393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d006f-a023-4857-a6dd-7c0f94c01769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:18.551568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-039d006f-a023-4857-a6dd-7c0f94c01769 None None] MFA Rules not processed for user `6c34613c69674b59957862a501334be0`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:18.552785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-039d006f-a023-4857-a6dd-7c0f94c01769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:18.617091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:18.649437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 658/1330] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:18.935567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:18.937128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:19.043092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744673ea-fe62-46c0-ad50-05d0556730cb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:19.063299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:19.118016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:19.118616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:19.118887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:19.119153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:19.120773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:19.231274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:19.232742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db3dff5-9d17-4438-9c0e-2e1239d18d6c service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:19.250451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 673/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:21:21.691912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:21.692464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:21.692464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:21.695501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 659/1332] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:21.700676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc171b67-8fb5-4131-8fcd-e26b7ce6151a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:21.701088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc171b67-8fb5-4131-8fcd-e26b7ce6151a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:21.701290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc171b67-8fb5-4131-8fcd-e26b7ce6151a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:21.766934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:21.770723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-bc171b67-8fb5-4131-8fcd-e26b7ce6151a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:21.772551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bc171b67-8fb5-4131-8fcd-e26b7ce6151a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:21.870382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 674/1333] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:22.151156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:22.152697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:22.254649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbd600-bdd1-4e81-b22b-3ef319d67a93 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:22.325508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:22.326094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:22.326366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:22.326632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:22.328195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:22.354672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:22.435446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:22.436841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dce299-530d-4d8c-882c-bdf0640e7afe service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:22.454702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 660/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:24.661923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:24.662466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:24.662466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:24.663781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 675/1335] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:24.669106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80c7b4-ae84-43aa-b709-2c3754358ad6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:24.669507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80c7b4-ae84-43aa-b709-2c3754358ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:24.669641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a80c7b4-ae84-43aa-b709-2c3754358ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:24.735380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1a80c7b4-ae84-43aa-b709-2c3754358ad6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:24.736628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1a80c7b4-ae84-43aa-b709-2c3754358ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:24.809416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:24.831139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 661/1336] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:25.122147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:25.123703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:25.223150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3b252-f072-4b4e-ab03-7101f5b92582 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:25.250711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:25.295970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:25.296590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:25.296861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:25.297124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:25.298705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:25.405813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:25.407229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ff5aa-a01e-436d-8d7e-43cae42f85b0 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:25.424481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 676/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:27.880094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:27.880626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:27.880719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:27.881999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 662/1338] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:27.887058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64b6cd-431a-4fe7-974d-4528d70ff47d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:27.887460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64b6cd-431a-4fe7-974d-4528d70ff47d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:27.887601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64b6cd-431a-4fe7-974d-4528d70ff47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:27.951302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5b64b6cd-431a-4fe7-974d-4528d70ff47d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:27.952523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5b64b6cd-431a-4fe7-974d-4528d70ff47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:27.961079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:28.047860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 677/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:28.327374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:28.328949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:28.432122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1104ab-dc10-45be-852e-0f96f8152032 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:28.503061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:28.503655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:28.503929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:28.504196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:28.506292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:28.540739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:28.613655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:28.615028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7033f70-6849-4226-8108-40c6fd69484e service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:28.633121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 663/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:32.654514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:32.655059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:32.655059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:32.656492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 678/1341] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:32.662009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6918-d306-45ff-8a01-1605161186c4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:32.662418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6918-d306-45ff-8a01-1605161186c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:32.662621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a6918-d306-45ff-8a01-1605161186c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:32.727755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-995a6918-d306-45ff-8a01-1605161186c4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:32.728904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-995a6918-d306-45ff-8a01-1605161186c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:32.813040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:32.825286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 664/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:33.121505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:33.123052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:33.225377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a68996-95b7-4f7b-8db0-ac9c2ffa884a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:33.263138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:33.298979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:33.299585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:33.299857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:33.300122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:33.302295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:33.412937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:33.414285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6fc2a-1de2-4791-833a-de613f4ebc5e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:33.433303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 679/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:36.224385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:36.224989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:36.224989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:36.226256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 665/1344] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:36.231404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3bfe-26fd-4df9-b022-5265ecfb1789 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:36.231845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3bfe-26fd-4df9-b022-5265ecfb1789 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:36.232024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e3bfe-26fd-4df9-b022-5265ecfb1789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:36.299204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-130e3bfe-26fd-4df9-b022-5265ecfb1789 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:36.300450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-130e3bfe-26fd-4df9-b022-5265ecfb1789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:36.318148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:36.398902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 680/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:36.678479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:36.680021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:36.790834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994cc60-387f-40ed-afa3-8ef355fc613b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:36.866724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:36.867335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:36.867616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:36.867891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:36.869527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:36.913738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:36.981389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:36.982745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb912be-c906-4b7d-9772-8007da86d9e5 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:37.002201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 666/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:41.505911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:41.506437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:41.506866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:41.509182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 681/1347] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:41.515544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df3d-3d04-4cad-a032-1aedad2c39a6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:41.515910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df3d-3d04-4cad-a032-1aedad2c39a6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:41.516106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359df3d-3d04-4cad-a032-1aedad2c39a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:41.592564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6359df3d-3d04-4cad-a032-1aedad2c39a6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:41.593989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6359df3d-3d04-4cad-a032-1aedad2c39a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:41.688791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:41.692908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 667/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:41.977291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:41.978847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:42.079276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9927ed4-faf9-4e39-801d-234750ee4312 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:42.126060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:42.150666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:42.151258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:42.151531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:42.151804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:42.153394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:42.258889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:42.260320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d0bc1-7408-4171-8759-4a6fbb6942ce service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:42.278434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 682/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:21:47.131553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe556-c2f2-485f-a9a2-c46aef3574ca None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:47.132560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe556-c2f2-485f-a9a2-c46aef3574ca None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:47.133070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fe556-c2f2-485f-a9a2-c46aef3574ca None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:47.136621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 668/1350] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:21:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:47.143039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b6c9-6e04-46db-abf4-660ddc6951ee None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:47.143864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b6c9-6e04-46db-abf4-660ddc6951ee None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:47.144381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330b6c9-6e04-46db-abf4-660ddc6951ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:47.240011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3330b6c9-6e04-46db-abf4-660ddc6951ee None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:47.240011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3330b6c9-6e04-46db-abf4-660ddc6951ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:47.265463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:47.421010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 683/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:21:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:51.596015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f97ee-7cfb-488a-9a23-3764d29632a3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:51.596859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f97ee-7cfb-488a-9a23-3764d29632a3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:51.596859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f97ee-7cfb-488a-9a23-3764d29632a3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:51.598689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 669/1352] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:51.603850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5eaf-b633-424c-9aba-55047dc7e17b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:51.604404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5eaf-b633-424c-9aba-55047dc7e17b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:51.605326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb5eaf-b633-424c-9aba-55047dc7e17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:51.689910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-35eb5eaf-b633-424c-9aba-55047dc7e17b None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:51.691211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-35eb5eaf-b633-424c-9aba-55047dc7e17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:51.764663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:51.802127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 684/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:21:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:53.148319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5251-d266-4ce8-aa81-f9e24e4a4971 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:53.148830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5251-d266-4ce8-aa81-f9e24e4a4971 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:53.148830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310c5251-d266-4ce8-aa81-f9e24e4a4971 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:53.150183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 670/1354] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:53.155270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8d9d1-1066-4f8b-acd4-fca1ffd56ce7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:53.155704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8d9d1-1066-4f8b-acd4-fca1ffd56ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:53.155817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf8d9d1-1066-4f8b-acd4-fca1ffd56ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:53.231904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5cf8d9d1-1066-4f8b-acd4-fca1ffd56ce7 None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:53.233106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5cf8d9d1-1066-4f8b-acd4-fca1ffd56ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:53.343399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 685/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:21:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:56.934225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee581b6-43fd-4360-84b1-1e8d9177aa78 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:56.934801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee581b6-43fd-4360-84b1-1e8d9177aa78 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:56.934801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee581b6-43fd-4360-84b1-1e8d9177aa78 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:56.936334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 671/1356] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:56.942601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6055ff-6a14-4635-a9e7-8bc00be26482 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:56.943088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6055ff-6a14-4635-a9e7-8bc00be26482 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:56.943335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6055ff-6a14-4635-a9e7-8bc00be26482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:56.980119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:57.021711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1c6055ff-6a14-4635-a9e7-8bc00be26482 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:57.022993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1c6055ff-6a14-4635-a9e7-8bc00be26482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:57.128076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 686/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:21:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:57.420465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:57.420817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:57.421019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:57.422322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 672/1358] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 22 12:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:21:57.431641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2f81-ab27-4d49-ad1b-2379e9fa216d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:57.432034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2f81-ab27-4d49-ad1b-2379e9fa216d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:57.432451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657f2f81-ab27-4d49-ad1b-2379e9fa216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:57.529732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-657f2f81-ab27-4d49-ad1b-2379e9fa216d None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:21:57.531072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-657f2f81-ab27-4d49-ad1b-2379e9fa216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:57.586110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:57.665765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 687/1359] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 22 12:21:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:21:57.666796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:21:57.668503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:57.824035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3301a-a219-46b9-9080-d2ab6b891dc3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:21:57.906417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:21:57.948787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:21:57.949399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:21:57.949655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:21:57.949926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:21:57.951493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:21:58.052836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:21:58.054444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1873341-0848-48b8-b652-251a8a73f6ec service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:21:58.079180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 673/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:22:00.486275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:00.486921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:00.486921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:00.490990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 688/1361] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:00.496186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f13480-68ef-41cb-8b75-6f2f23940951 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:00.496704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f13480-68ef-41cb-8b75-6f2f23940951 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:00.496873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f13480-68ef-41cb-8b75-6f2f23940951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:00.571889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-44f13480-68ef-41cb-8b75-6f2f23940951 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:00.573238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-44f13480-68ef-41cb-8b75-6f2f23940951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:00.612360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:00.675421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 674/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:00.965295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:00.966816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:01.139089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d3c21-4f9c-418c-9d35-189cbbb0a07c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:01.150126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:01.265853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:01.266487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:01.266761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:01.267032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:01.268643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:01.361244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:01.362827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ab874-82f9-4c6f-91d5-5ad6ef6672ca service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:01.381172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 689/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:22:03.890637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81adae-3770-439d-9616-ecec3501f5fd None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:03.891122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81adae-3770-439d-9616-ecec3501f5fd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:03.891202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81adae-3770-439d-9616-ecec3501f5fd None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:03.892540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 675/1364] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:03.897860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d11e-e3ef-4989-8b4c-4c5f227d58a4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:03.898260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d11e-e3ef-4989-8b4c-4c5f227d58a4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:03.898425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8d11e-e3ef-4989-8b4c-4c5f227d58a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:03.914869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:03.967522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-41c8d11e-e3ef-4989-8b4c-4c5f227d58a4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:03.968753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-41c8d11e-e3ef-4989-8b4c-4c5f227d58a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:04.075198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 690/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:22:04.370596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:04.371176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:04.371651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:04.373434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 676/1366] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 22 12:22:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:04.379271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3341-382e-406b-81b2-59edd38fd311 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:04.379565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3341-382e-406b-81b2-59edd38fd311 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:04.379839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea3341-382e-406b-81b2-59edd38fd311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:04.458068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-23ea3341-382e-406b-81b2-59edd38fd311 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:04.459275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-23ea3341-382e-406b-81b2-59edd38fd311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:04.496338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:04.581291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 691/1367] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 22 12:22:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:04.589190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:04.590699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:04.745455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe09852-05ae-4085-8d66-6cd240b56edc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:04.766966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:04.871943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:04.871943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:04.871943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:04.871943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:04.872447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:05.095822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:05.098028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c44a08-cfdf-4b15-bb6f-22d5fcd94e3e service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:05.107515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b733b-01b4-4a88-8c26-7ea324252618 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:05.107988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b733b-01b4-4a88-8c26-7ea324252618 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:05.108092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477b733b-01b4-4a88-8c26-7ea324252618 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:05.109726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 692/1368] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:22:05.114421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822ddc8-78d0-48b1-92a1-7484d30078e3 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:05.114703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822ddc8-78d0-48b1-92a1-7484d30078e3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:05.114975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822ddc8-78d0-48b1-92a1-7484d30078e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:05.118725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 677/1369] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:05.196827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c822ddc8-78d0-48b1-92a1-7484d30078e3 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:05.198455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c822ddc8-78d0-48b1-92a1-7484d30078e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:05.322875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:05.340352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 693/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:22:05.665934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a6b12-091b-4094-b34e-31af502a362f None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:05.665934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a6b12-091b-4094-b34e-31af502a362f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:05.665934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a6b12-091b-4094-b34e-31af502a362f None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:05.669511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 678/1371] 10.222.0.33 () {54 vars in 963 bytes} [Thu Sep 22 12:22:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:05.670947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5842e-2e2b-4d85-be80-9a5d7c28928f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:05.671354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5842e-2e2b-4d85-be80-9a5d7c28928f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:05.671533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5842e-2e2b-4d85-be80-9a5d7c28928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:05.760091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7fa5842e-2e2b-4d85-be80-9a5d7c28928f None None] MFA Rules not processed for user `e10e2b0b66754f128cddc94ca3daf101`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:05.762383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7fa5842e-2e2b-4d85-be80-9a5d7c28928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:05.849776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 694/1372] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Sep 22 12:22:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:05.857684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a6b12-091b-4094-b34e-31af502a362f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:05.859400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-184a6b12-091b-4094-b34e-31af502a362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:05.895581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:05.977687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a6b12-091b-4094-b34e-31af502a362f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:06.081184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:06.082848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:06.083539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:06.084261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:06.087426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:06.205665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:06.239411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:06.240832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100a806-0cd4-4503-aa3a-432f44e0e2b6 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:06.260614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 679/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:07.365051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:07.365622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:07.365622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:07.366933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 695/1374] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:22:07.372108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a355-49ea-42fa-a0a9-7e7813d778c5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:07.372564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a355-49ea-42fa-a0a9-7e7813d778c5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:07.372693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4a355-49ea-42fa-a0a9-7e7813d778c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:07.437991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-07a4a355-49ea-42fa-a0a9-7e7813d778c5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:07.439217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-07a4a355-49ea-42fa-a0a9-7e7813d778c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:07.535329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 680/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:07.829387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:07.830938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:07.932664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:07.997957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f5f7d-7088-4bd6-8b0c-c7340f7d0eb1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:08.111680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:08.112289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:08.112584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:08.112848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:08.114448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:08.165132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:08.208811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:08.210711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f858bf8-f249-406b-9a52-2ef09adba1bc service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:08.229014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 696/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:10.442196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:10.442783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:10.442783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:10.444229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 681/1377] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:10.449301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7dc8-a958-4d58-95c1-c3415c71c21c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:10.449724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7dc8-a958-4d58-95c1-c3415c71c21c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:10.449820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7dc8-a958-4d58-95c1-c3415c71c21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:10.516916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-695e7dc8-a958-4d58-95c1-c3415c71c21c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:10.518108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-695e7dc8-a958-4d58-95c1-c3415c71c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:10.625705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 697/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:22:10.917154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:10.918642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:10.957818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:11.073768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01263f-9c24-4756-866d-af646848ed1e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:11.188336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:11.194346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:11.194948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:11.195223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:11.195490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:11.197094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:11.288399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:11.289974 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d78961-caae-4ba7-87c4-e8c548336b55 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:11.308781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 682/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:13.529972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:13.530550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:13.530550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:13.531971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 698/1380] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:13.537577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb57db-b0bb-4b4e-ba2d-c6e0a667fc92 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:13.538009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb57db-b0bb-4b4e-ba2d-c6e0a667fc92 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:13.538099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb57db-b0bb-4b4e-ba2d-c6e0a667fc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:13.619717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b0fb57db-b0bb-4b4e-ba2d-c6e0a667fc92 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:13.622558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b0fb57db-b0bb-4b4e-ba2d-c6e0a667fc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:13.734523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:13.801920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 683/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:14.139424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:14.141044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:14.173032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:14.306712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf69006-8cdb-42d4-b0ff-d9b99839c713 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:14.407108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:14.425251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:14.426354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:14.426628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:14.426893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:14.428513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:14.523111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:14.525134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66e6d3-6431-4599-a27b-0ed445641fd2 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:14.547036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 699/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:16.761736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:16.762387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:16.762387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:16.763479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 684/1383] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:16.769175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b32e9-34c1-4d3e-80e3-c8e81d0affef None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:16.769614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b32e9-34c1-4d3e-80e3-c8e81d0affef None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:16.769845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b32e9-34c1-4d3e-80e3-c8e81d0affef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:16.837802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-cc8b32e9-34c1-4d3e-80e3-c8e81d0affef None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:16.839006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cc8b32e9-34c1-4d3e-80e3-c8e81d0affef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:16.898859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:16.944993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 700/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:17.243732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:17.245329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:17.403847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab115c1-0bad-4101-b3cd-fa9ebdf47fa2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:17.458130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:17.521964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:17.522453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:17.522742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:17.523020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:17.524791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:17.618732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:17.620707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64488bc-5c65-4c2d-8f1c-ae8cf95a328b service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:17.642267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 685/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:22:19.908220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:19.908832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:19.908832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:19.910753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 701/1386] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:19.916117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedea856-037c-42ba-838c-47d973762358 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:19.916642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedea856-037c-42ba-838c-47d973762358 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:19.916801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedea856-037c-42ba-838c-47d973762358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:19.983051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-bedea856-037c-42ba-838c-47d973762358 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:19.984254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bedea856-037c-42ba-838c-47d973762358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:19.992566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:20.103646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 686/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:20.404508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:20.406024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:20.554928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c665a-7b08-48a1-a0d4-134ce874899b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:20.599956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:20.665792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:20.666309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:20.666582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:20.666846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:20.668433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:20.756670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:20.758252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b103cf-f542-4635-a1ca-8289b883bec9 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:20.776156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 702/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:22.586648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179cb17-1577-4e59-ae89-ab0cb1302b77 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:22.587249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179cb17-1577-4e59-ae89-ab0cb1302b77 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:22.587249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179cb17-1577-4e59-ae89-ab0cb1302b77 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:22.588744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 687/1389] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:22.593765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0441-bc18-4e62-9753-1b83eca12230 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:22.594225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0441-bc18-4e62-9753-1b83eca12230 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:22.594331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd0441-bc18-4e62-9753-1b83eca12230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:22.656961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:22.660108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c2cd0441-bc18-4e62-9753-1b83eca12230 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:22.662112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c2cd0441-bc18-4e62-9753-1b83eca12230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:22.765336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 703/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:23.013249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8d28-ba4b-4164-bcc7-2abba0412cb8 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.013536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8d28-ba4b-4164-bcc7-2abba0412cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.013811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5a8d28-ba4b-4164-bcc7-2abba0412cb8 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.015150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 688/1391] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:23.021273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.021560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.021831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.073331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742ddca-2336-4c56-94f1-105405680700 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.073610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742ddca-2336-4c56-94f1-105405680700 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.073877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742ddca-2336-4c56-94f1-105405680700 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.075626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 689/1392] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 22 12:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:22:23.079896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.080165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.080474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.093876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:23.095088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.157187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] MFA Rules not processed for user `628750c611c24e8b9b4c0e66d2e3282e`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:23.158445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.170191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:23.194702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 704/1393] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:23.233389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 690/1394] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 22 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:22:23.242339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:23.243670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.338892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a42a6-8344-4b5c-9b07-744147471504 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:23.414394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:23.414915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.415186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.415449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.417045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.465980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:23.491926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:23.493277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.497252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:23.529693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:23.531126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f104f1-5619-4a82-8378-c514e681c370 service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:23.551677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 705/1395] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:23.610754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334dace-7ffb-4da3-a2ac-39d6a0f55a47 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:23.719026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:23.726233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:23.726834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:23.727106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:23.727370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:23.728976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:23.834317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:23.835940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262b4c9-ad40-463d-83a5-065b2442b248 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:23.857305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 691/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:22:24.319847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052027-9ac5-4341-a4cd-1c9506cc56e5 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:24.320339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052027-9ac5-4341-a4cd-1c9506cc56e5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:24.320453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e052027-9ac5-4341-a4cd-1c9506cc56e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:24.386937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4e052027-9ac5-4341-a4cd-1c9506cc56e5 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:24.388120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4e052027-9ac5-4341-a4cd-1c9506cc56e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:24.480660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 706/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:24.522475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:24.522828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:24.523185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:24.524615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 692/1398] 10.222.0.33 () {54 vars in 943 bytes} [Thu Sep 22 12:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:24.531226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35676e-3a34-41f1-993d-415b13603c3d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:24.531505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35676e-3a34-41f1-993d-415b13603c3d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:24.531770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb35676e-3a34-41f1-993d-415b13603c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:24.552386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:24.608830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fb35676e-3a34-41f1-993d-415b13603c3d None None] MFA Rules not processed for user `40dc194ca31947339986b511f4271498`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:24.610028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb35676e-3a34-41f1-993d-415b13603c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:24.717361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 707/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Thu Sep 22 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:24.727248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:24.728783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:24.850395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:24.876136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e3e686-3c92-4db7-9e81-de9ba1b75ef8 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:24.997033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:24.997638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:24.997910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:24.998177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:24.999776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:25.089406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:25.091018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ac0f0-dbd1-4bb3-8931-adc283374e36 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:25.099488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 693/1400] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:22:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:22:26.089033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:26.089593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:26.089593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:26.090915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 708/1401] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:26.096204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b4a03-c931-42b5-bc9f-0a0d74fe0a5d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:26.097442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b4a03-c931-42b5-bc9f-0a0d74fe0a5d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:26.097868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b4a03-c931-42b5-bc9f-0a0d74fe0a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:26.122346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:26.170954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-cd4b4a03-c931-42b5-bc9f-0a0d74fe0a5d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:26.172186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cd4b4a03-c931-42b5-bc9f-0a0d74fe0a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:26.279741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 694/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:26.568587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:26.570465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:26.653093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:26.731466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89587d84-e00a-44ad-861d-f8bc24372719 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:26.851712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:26.852350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:26.852618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:26.852886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:26.854470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:26.900150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:26.953374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:26.954987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284025c6-d967-4ecb-9346-93619c7c7cf2 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:26.974624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 709/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:29.183038 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:29.183640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:29.183640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:29.185059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 695/1404] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:29.190200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02071-aaaa-4404-b8ae-eba70869d56e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:29.191181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02071-aaaa-4404-b8ae-eba70869d56e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:29.191278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab02071-aaaa-4404-b8ae-eba70869d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:29.263570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6ab02071-aaaa-4404-b8ae-eba70869d56e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:29.265104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ab02071-aaaa-4404-b8ae-eba70869d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:29.375630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 710/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:29.665401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:29.666935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:29.775841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:29.840616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2206357-443b-4750-b7ac-5f758ca3560c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:29.969081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:29.969681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:29.969960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:29.970229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:29.972740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:30.038303 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:30.072940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:30.074635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b5bd-0f97-42cf-a30c-a9c27af7f5b0 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:30.092714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 696/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:32.307858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:32.308405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:32.308405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:32.313495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 711/1407] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:22:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:32.314862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0402d-078f-49c4-9222-d302602c9d18 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:32.315262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0402d-078f-49c4-9222-d302602c9d18 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:32.315942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0402d-078f-49c4-9222-d302602c9d18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:32.391452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ecc0402d-078f-49c4-9222-d302602c9d18 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:32.392706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ecc0402d-078f-49c4-9222-d302602c9d18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:32.497668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 697/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:22:32.789323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:32.790867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:32.809585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:32.965409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a7b77-1a83-4967-a032-5b3b1e8d9d47 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:33.066264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:33.097789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:33.098400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:33.098682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:33.098955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:33.100598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:33.197645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:33.199325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c467a-1c9e-4af7-987f-ccecf009f380 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:33.219839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 712/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:42.496129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=84 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:43.217473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=56 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:47.670480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eeb3c-7edb-4511-a6f9-e43f5ae13b0c None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:47.671065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eeb3c-7edb-4511-a6f9-e43f5ae13b0c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:47.671065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8eeb3c-7edb-4511-a6f9-e43f5ae13b0c None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:47.672372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 698/1410] 10.222.0.33 () {54 vars in 918 bytes} [Thu Sep 22 12:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:47.677505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48653e-f5e0-4d51-a7db-80375b7d8af3 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:47.677937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48653e-f5e0-4d51-a7db-80375b7d8af3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:47.678048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f48653e-f5e0-4d51-a7db-80375b7d8af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:47.751495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7f48653e-f5e0-4d51-a7db-80375b7d8af3 None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:47.752714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7f48653e-f5e0-4d51-a7db-80375b7d8af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:47.861071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 713/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 22 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:47.875296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92fe0c-52e9-4e5a-bcf6-b0a61e666306 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:47.875569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92fe0c-52e9-4e5a-bcf6-b0a61e666306 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:47.875836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92fe0c-52e9-4e5a-bcf6-b0a61e666306 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:47.877167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 699/1412] 10.222.0.33 () {54 vars in 918 bytes} [Thu Sep 22 12:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:47.882302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031edc-6c91-49e0-97ef-16b97b3ca99e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:47.882583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031edc-6c91-49e0-97ef-16b97b3ca99e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:47.882849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13031edc-6c91-49e0-97ef-16b97b3ca99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:47.954645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:47.958286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-13031edc-6c91-49e0-97ef-16b97b3ca99e None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:47.960191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-13031edc-6c91-49e0-97ef-16b97b3ca99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:48.073521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 714/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Thu Sep 22 12:22:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:50.646314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:50.646851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:50.646851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:50.648201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 700/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:50.653396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efab8b-97ae-4f3a-b95c-eb9a83fb3e31 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:50.653795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efab8b-97ae-4f3a-b95c-eb9a83fb3e31 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:50.653935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efab8b-97ae-4f3a-b95c-eb9a83fb3e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:50.717662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c1efab8b-97ae-4f3a-b95c-eb9a83fb3e31 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:50.718843 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c1efab8b-97ae-4f3a-b95c-eb9a83fb3e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:50.771563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:50.812162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 715/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:51.101063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:51.102688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:51.243048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0f1fd-c2f4-46f1-96b9-9de22fc6f761 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:51.313008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:51.352144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:51.352693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:51.352963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:51.353223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:51.354788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:51.441468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:51.443021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c06b28-0792-4fc9-8ffe-cc9913fa0626 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:51.460388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 701/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:52.588984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:52.589558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:52.589558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:52.590843 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 716/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:52.596920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87817582-17c0-480f-9c3e-4d37f6e35341 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:52.597664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87817582-17c0-480f-9c3e-4d37f6e35341 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:52.597838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87817582-17c0-480f-9c3e-4d37f6e35341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:52.671000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-87817582-17c0-480f-9c3e-4d37f6e35341 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:52.672176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-87817582-17c0-480f-9c3e-4d37f6e35341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:52.698969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:52.767233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 702/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:52.923029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:52.924563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:53.068451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b344eb4-0a6e-4c6e-9814-10372f0edf32 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:53.107548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:53.181551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:53.182072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:53.182347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:53.182617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:53.184207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:53.271461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:53.273094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5794e312-fa1c-45d6-817d-0a50f2aee765 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:53.291124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 717/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:54.436737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:54.437324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:54.437324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:54.438609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 703/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:54.445497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083b5c-b2f4-429a-ae20-55ffaa57bf2d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:54.445908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083b5c-b2f4-429a-ae20-55ffaa57bf2d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:54.446044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083b5c-b2f4-429a-ae20-55ffaa57bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:54.498910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:54.521132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5f083b5c-b2f4-429a-ae20-55ffaa57bf2d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:54.522315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5f083b5c-b2f4-429a-ae20-55ffaa57bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:54.616790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 718/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:22:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:54.767090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:54.768537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:54.910345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211a2a4-a0b1-41be-bee6-387b9c718406 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:54.923012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:55.023585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:55.024110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:55.024412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:55.024680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:55.026244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:55.113019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:55.114585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722d8c2-02ec-4811-a283-3251e5f2e399 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:55.132558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 704/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:56.234628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:56.235141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:56.235141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:56.236489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 719/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:56.243435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c6af-672d-4dea-b2f9-934f95bdd1be None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:56.243877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c6af-672d-4dea-b2f9-934f95bdd1be None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:56.243988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c6af-672d-4dea-b2f9-934f95bdd1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:56.263302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:56.320064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-bdf3c6af-672d-4dea-b2f9-934f95bdd1be None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:56.321296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bdf3c6af-672d-4dea-b2f9-934f95bdd1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:56.415000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 705/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:56.564112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:56.566196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:56.673591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:56.709137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a44773-7745-4917-a52d-6f49e0580f94 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:56.818702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:56.819236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:56.819542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:56.819810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:56.821421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:56.896102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:56.909666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:56.911801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd29c843-779f-4a24-860d-daf8b3f24c88 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:56.929938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 720/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:58.036279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06076271-9623-4ea1-a503-e308cf978444 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:58.036860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06076271-9623-4ea1-a503-e308cf978444 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:58.036860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06076271-9623-4ea1-a503-e308cf978444 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:58.038196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 706/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:58.044385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747ee7-ff31-4873-ab87-2b0f236b4960 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:58.044786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747ee7-ff31-4873-ab87-2b0f236b4960 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:58.044924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747ee7-ff31-4873-ab87-2b0f236b4960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:58.118437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-10747ee7-ff31-4873-ab87-2b0f236b4960 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:58.119616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-10747ee7-ff31-4873-ab87-2b0f236b4960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:58.213624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 721/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:22:58.364031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06076271-9623-4ea1-a503-e308cf978444 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:22:58.365576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-06076271-9623-4ea1-a503-e308cf978444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:58.449170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:58.509372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06076271-9623-4ea1-a503-e308cf978444 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:22:58.619623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:22:58.620216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:58.620521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:58.620793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:58.622376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:58.670284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:22:58.710590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:22:58.712172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3039e363-fe7c-4fb4-90a3-467c0504ea4a service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:22:58.731731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 707/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:22:59.779630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:59.780189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:59.780189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:59.781646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 722/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:22:59.788658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7069-a7cd-40ac-b911-35044121d9c5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:22:59.789068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7069-a7cd-40ac-b911-35044121d9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:22:59.789200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7069-a7cd-40ac-b911-35044121d9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:22:59.871386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3bde7069-a7cd-40ac-b911-35044121d9c5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:22:59.872646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3bde7069-a7cd-40ac-b911-35044121d9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:22:59.968190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 708/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:00.119133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:00.120724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:00.168869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:00.261614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b602ed-fe32-48cd-8609-30e21b17e3ef None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:00.371813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:00.372365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:00.372642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:00.372911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:00.374542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:00.381627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:00.462671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:00.464262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122bea1-e9f1-4237-9566-3a0890064314 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:00.482576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 723/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:01.527444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:01.528014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:01.528014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:01.529354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 709/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:01.535847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bfe-f7ba-4e40-8fa9-e48b88f9eef4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:01.536258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bfe-f7ba-4e40-8fa9-e48b88f9eef4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:01.536535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59bfe-f7ba-4e40-8fa9-e48b88f9eef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:01.611465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-82c59bfe-f7ba-4e40-8fa9-e48b88f9eef4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:01.612689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-82c59bfe-f7ba-4e40-8fa9-e48b88f9eef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:01.684204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:01.712953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 724/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:01.866086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:01.867596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:01.885869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:02.010107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fba447-01a4-400b-b8f7-39bad2723b0e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:02.092922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:02.118787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:02.119322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:02.119594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:02.119860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:02.121447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:02.206793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:02.208810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb2861-4dce-4751-93dd-435b3df02e0b service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:02.230547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 710/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:03.278508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:03.279048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:03.279048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:03.280279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 725/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:03.286932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765f94-a516-410d-a857-9cc2834ee3c4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:03.287368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765f94-a516-410d-a857-9cc2834ee3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:03.287478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b765f94-a516-410d-a857-9cc2834ee3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:03.362445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9b765f94-a516-410d-a857-9cc2834ee3c4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:03.363652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9b765f94-a516-410d-a857-9cc2834ee3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:03.403392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:03.455802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 711/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:03.604240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:03.605834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:03.748948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f869886-c8ff-45ef-80a2-55e5f90889f3 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:03.803033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:03.861372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:03.861985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:03.862263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:03.862530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:03.864758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:03.951750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:03.953369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68203438-5c80-42a9-aee5-762d8186f320 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:03.971149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 726/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:05.009684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:05.010248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:05.010248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:05.011552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 712/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:05.017795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4b2fa-d871-41fa-a87f-d9cd86f6e5b4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:05.018273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4b2fa-d871-41fa-a87f-d9cd86f6e5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:05.018388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4b2fa-d871-41fa-a87f-d9cd86f6e5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:05.093821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-07d4b2fa-d871-41fa-a87f-d9cd86f6e5b4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:05.094926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-07d4b2fa-d871-41fa-a87f-d9cd86f6e5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:05.096911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:05.189011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 727/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:05.340405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:05.341983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:05.483737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adf87-2b42-4b62-8113-549d87834f7c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:05.512131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:05.599819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:05.600447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:05.600723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:05.600989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:05.602560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:05.688971 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:05.690531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee9660-9db3-4889-a35f-78940f2abc1b service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:05.708540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 713/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:07.763193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:07.763784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:07.763784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:07.764993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 728/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:07.770613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00d86-4ede-4b3c-8383-8638aa5ef158 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:07.771018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00d86-4ede-4b3c-8383-8638aa5ef158 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:07.771195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f00d86-4ede-4b3c-8383-8638aa5ef158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:07.811256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:07.839751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-00f00d86-4ede-4b3c-8383-8638aa5ef158 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:07.840999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-00f00d86-4ede-4b3c-8383-8638aa5ef158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:07.946283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 714/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:08.230381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:08.231899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:08.331303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c50fad-c7ca-4885-b515-f44ba2141d5a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:08.363074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:08.404132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:08.404758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:08.405030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:08.405294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:08.406853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:08.513913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:08.516076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44730690-31b8-49f4-bef3-3622c12370a2 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:08.534809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 729/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:10.812870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:10.813453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:10.813453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:10.814823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 715/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:10.820899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b79e-a9be-4950-a18a-3fb5dc6e5f11 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:10.821359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b79e-a9be-4950-a18a-3fb5dc6e5f11 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:10.821499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513b79e-a9be-4950-a18a-3fb5dc6e5f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:10.887685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7513b79e-a9be-4950-a18a-3fb5dc6e5f11 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:10.888928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7513b79e-a9be-4950-a18a-3fb5dc6e5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:10.901382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:11.099704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 730/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:11.383242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:11.384863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:11.486914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441e5b1-5a89-4068-8589-7768ac9a6618 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:11.499343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:11.558587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:11.559182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:11.559453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:11.559722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:11.561365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:11.668548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:11.669903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ede77f-7616-47d4-b240-32c8da7a4336 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:11.687172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 716/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:14.122622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:14.123222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:14.123222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:14.124523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 731/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:14.129825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144c4d-a8f5-4ae5-b87a-a594674cd8d0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:14.130254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144c4d-a8f5-4ae5-b87a-a594674cd8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:14.130357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8144c4d-a8f5-4ae5-b87a-a594674cd8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:14.177795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:14.195246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a8144c4d-a8f5-4ae5-b87a-a594674cd8d0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:14.196616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a8144c4d-a8f5-4ae5-b87a-a594674cd8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:14.290773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 717/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:14.576929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:14.578628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:14.679361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea5825-7ffa-476d-822a-1b2028b8f2e1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:14.718462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:14.753467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:14.754562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:14.754832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:14.755096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:14.756770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:14.862708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:14.864141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3305435-7b96-4e1d-84a1-7fbd009e24e3 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:14.882168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 732/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:17.139360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:17.139882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:17.139882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:17.141249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 718/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:17.146601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09917a1-79a2-4ff3-a375-cb8fa144ce2c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:17.147039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09917a1-79a2-4ff3-a375-cb8fa144ce2c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:17.147135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09917a1-79a2-4ff3-a375-cb8fa144ce2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:17.218550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c09917a1-79a2-4ff3-a375-cb8fa144ce2c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:17.219749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c09917a1-79a2-4ff3-a375-cb8fa144ce2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:17.240526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:17.314356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 733/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:17.603615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:17.605162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:17.704549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d376c86-3d32-42e6-b401-dc0a9c513878 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:17.725480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:17.777388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:17.777976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:17.778249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:17.778513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:17.780065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:17.886083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:17.887497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0f1b4-47a9-4880-bf5f-a0adea802f36 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:17.906734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 719/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:20.109596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:20.110167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:20.110167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:20.111451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 734/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:20.117474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79602f7-ce99-422f-b161-820d6cebe75d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:20.117903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79602f7-ce99-422f-b161-820d6cebe75d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:20.118006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79602f7-ce99-422f-b161-820d6cebe75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:20.181441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:20.184431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a79602f7-ce99-422f-b161-820d6cebe75d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:20.186064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a79602f7-ce99-422f-b161-820d6cebe75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:20.280282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 720/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:20.564780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:20.566299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:20.666738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd30433-0b02-4cd3-a886-5737cc07a32e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:20.714405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:20.738786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:20.739399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:20.739669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:20.739930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:20.741522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:20.848615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:20.850041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53149be-9e6d-4589-a3f6-c07ff4e7ba7f service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:20.867818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 735/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:23.053399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:23.053931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:23.053931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:23.055255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 721/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:23.060874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303c43-7204-4d7b-a1ea-0aaf2ab8c373 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:23.061318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303c43-7204-4d7b-a1ea-0aaf2ab8c373 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:23.061414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63303c43-7204-4d7b-a1ea-0aaf2ab8c373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:23.126388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-63303c43-7204-4d7b-a1ea-0aaf2ab8c373 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:23.127502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-63303c43-7204-4d7b-a1ea-0aaf2ab8c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:23.156134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:23.225289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 736/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:23.506492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:23.508020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:23.611016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbce41-55ff-4024-a435-16ee09f5cecf None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:23.640075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:23.685082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:23.685682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:23.685957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:23.686227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:23.687814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:23.794655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:23.796007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46ef43-3f58-4351-95ac-ef6977f3591c service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:23.814224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 722/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:26.062202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:26.062797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:26.062797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:26.064146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 737/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:26.069319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc618e-e2c3-48f4-96a4-3181f27ada0e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:26.069743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc618e-e2c3-48f4-96a4-3181f27ada0e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:26.069868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc618e-e2c3-48f4-96a4-3181f27ada0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:26.137884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4afc618e-e2c3-48f4-96a4-3181f27ada0e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:26.138990 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4afc618e-e2c3-48f4-96a4-3181f27ada0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:26.147119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:26.233208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 723/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:26.513803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:26.515393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:26.617549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ad0a-c2f8-4bd3-a5ea-258fcb6ebcf9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:26.670946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:26.689369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:26.689967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:26.690239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:26.690503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:26.692728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:26.798416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:26.799750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcec52-0a65-4836-8d73-20bae6877093 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:26.817735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 738/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:28.997258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:28.997784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:28.997784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:28.999033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 724/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:29.004333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315650f-3352-447c-8715-36469ce3f216 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:29.005253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315650f-3352-447c-8715-36469ce3f216 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:29.005379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315650f-3352-447c-8715-36469ce3f216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:29.068837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2315650f-3352-447c-8715-36469ce3f216 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:29.070024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2315650f-3352-447c-8715-36469ce3f216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:29.107195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:29.163575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 739/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:29.444513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:29.446103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:29.546186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a92e-e021-4773-a0a4-e8531c4dc79e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:29.583626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:29.618969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:29.619563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:29.619836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:29.620101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:29.621686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:29.727682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:29.729068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f55cbb-53cb-42e9-bdee-05a4f5b5be4b service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:29.747170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 725/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:31.895865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:31.896457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:31.896457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:31.897895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 740/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:31.903581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd80da-7de7-4fda-802a-3dffd30866ba None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:31.904178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd80da-7de7-4fda-802a-3dffd30866ba None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:31.904556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd80da-7de7-4fda-802a-3dffd30866ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:31.974217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c4fd80da-7de7-4fda-802a-3dffd30866ba None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:31.975431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c4fd80da-7de7-4fda-802a-3dffd30866ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:31.993097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:32.070016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 726/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:32.353071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:32.354595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:32.455659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6239d-13a3-4e59-ba86-0ca8404f7030 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:32.526346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:32.533110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:32.533707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:32.533974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:32.534234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:32.536389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:32.643746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:32.645627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fcba6-fcc3-401b-b5ef-d9aac50108c8 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:32.663340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 741/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:34.851635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:34.852177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:34.852177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:34.853520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 727/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:34.858726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2555-90ee-4410-904b-6d6cf694f310 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:34.859126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2555-90ee-4410-904b-6d6cf694f310 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:34.859296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe2555-90ee-4410-904b-6d6cf694f310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:34.924935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-94fe2555-90ee-4410-904b-6d6cf694f310 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:34.926139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-94fe2555-90ee-4410-904b-6d6cf694f310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:34.973753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:35.022700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 742/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:35.312920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:35.314439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:35.416063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb16d46-e6b1-40fb-b857-b1c86d4f9113 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:35.462712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:35.488575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:35.489212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:35.489484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:35.489748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:35.491297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:35.597805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:35.600122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c945c7-f9fd-4ffc-975f-f8a61af5e875 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:35.617571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 728/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:45.021162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=22 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:45.615302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=66 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:55.510788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4697f-cf76-458a-b296-7d87744f24f0 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:55.511364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4697f-cf76-458a-b296-7d87744f24f0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:55.511364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4697f-cf76-458a-b296-7d87744f24f0 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:55.512731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 743/1471] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:55.517823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6552-2fa1-4e10-94e0-1b4c26616ee3 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:55.518257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6552-2fa1-4e10-94e0-1b4c26616ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:55.518368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a6552-2fa1-4e10-94e0-1b4c26616ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:55.584246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-249a6552-2fa1-4e10-94e0-1b4c26616ee3 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:55.585461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-249a6552-2fa1-4e10-94e0-1b4c26616ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:55.679525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 729/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:23:55.976283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:55.976585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:55.976855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:55.978223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 744/1473] 10.222.0.33 () {54 vars in 962 bytes} [Thu Sep 22 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:55.983050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3b8dd-c5e7-4036-a8d9-dc71e570efdf None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:55.983336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3b8dd-c5e7-4036-a8d9-dc71e570efdf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:55.983606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3b8dd-c5e7-4036-a8d9-dc71e570efdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:56.058571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c9c3b8dd-c5e7-4036-a8d9-dc71e570efdf None None] MFA Rules not processed for user `628750c611c24e8b9b4c0e66d2e3282e`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:56.059774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c9c3b8dd-c5e7-4036-a8d9-dc71e570efdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:56.079597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:56.135538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 730/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Thu Sep 22 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:56.143312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:56.144937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:56.246056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6545f5-76e2-4afa-9a90-a613050f0c16 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:56.317523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:56.318117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:56.318391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:56.318658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:56.320221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:56.372806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:56.426485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:56.427835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db271e9f-8b98-4902-94a7-248a3828a070 service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:56.445613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 745/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:23:58.674142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:58.674674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:58.674674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:58.676465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 731/1476] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:23:58.681801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406abbe-53b2-4767-b001-3b7a33072745 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:58.682203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406abbe-53b2-4767-b001-3b7a33072745 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:58.682349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406abbe-53b2-4767-b001-3b7a33072745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:58.748494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0406abbe-53b2-4767-b001-3b7a33072745 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:23:58.749694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0406abbe-53b2-4767-b001-3b7a33072745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:58.843052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 746/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:23:59.123812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:23:59.125375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:23:59.245958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ede71-6c0a-4f12-9747-cea29a4779be None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:23:59.293489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:23:59.336499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:23:59.337096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:23:59.337370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:23:59.337636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:23:59.338247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:23:59.339955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:23:59.339955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:23:59.339955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:23:59.341117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f510-f2ba-430f-8923-8100465191a1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:23:59.360335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 732/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Thu Sep 22 12:23:59 2022] GET /identity/v3/projects => generated 2553 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:24:01.533703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:01.534268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:01.534268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:01.535608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 747/1479] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:24:01.540883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66368b-6055-447d-be2d-b49fd936296f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:01.541330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66368b-6055-447d-be2d-b49fd936296f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:01.541444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f66368b-6055-447d-be2d-b49fd936296f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:01.607707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0f66368b-6055-447d-be2d-b49fd936296f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:24:01.608925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0f66368b-6055-447d-be2d-b49fd936296f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:01.702526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 733/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:24:01.998472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:24:02.000068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:02.010741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:02.153461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c3b1df-de91-4a10-a2a0-95df30898b41 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:24:02.231593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:02.263676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:24:02.264317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:02.264594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:02.264855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:02.266445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:02.353734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:24:02.355318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05aeda8-dad4-4bd1-a3e1-d6873693a9e6 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:24:02.373318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 748/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:24:04.698396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:04.698940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:04.698940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:04.700251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 734/1482] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:24:04.705406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95dea-91ef-44ec-8b38-50be625865b8 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:04.705816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95dea-91ef-44ec-8b38-50be625865b8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:04.705944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed95dea-91ef-44ec-8b38-50be625865b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:04.770062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3ed95dea-91ef-44ec-8b38-50be625865b8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:24:04.771241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ed95dea-91ef-44ec-8b38-50be625865b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:04.816007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:04.866740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 749/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:24:05.153089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:24:05.154673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:05.180023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:05.259790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b328d13-e09a-41ac-bbe0-ac33f5721191 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:24:05.330642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:24:05.331236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:05.331510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:05.331780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:05.333394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:05.406555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:05.440814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:24:05.442261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ce546-c4c5-41be-862e-09c55ab977ef service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:24:05.461652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 735/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:24:07.670294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaee12-8673-480f-8104-d8699298206c None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:07.670808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaee12-8673-480f-8104-d8699298206c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:07.670808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aaee12-8673-480f-8104-d8699298206c None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:07.672142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 750/1485] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:24:07.677995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650fb9-876d-4c0e-9dbe-dda88782685a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:07.678424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650fb9-876d-4c0e-9dbe-dda88782685a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:07.678528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72650fb9-876d-4c0e-9dbe-dda88782685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:07.743044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-72650fb9-876d-4c0e-9dbe-dda88782685a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:24:07.744230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-72650fb9-876d-4c0e-9dbe-dda88782685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:07.838125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 736/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:24:08.138758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaee12-8673-480f-8104-d8699298206c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:24:08.140269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-50aaee12-8673-480f-8104-d8699298206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:08.240053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aaee12-8673-480f-8104-d8699298206c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:24:08.310125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:08.311567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:24:08.312242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:08.312729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:08.313202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:08.315034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:08.423707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:24:08.425071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b8204d-b407-451e-a163-0bb48125d2c1 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:24:08.443751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 751/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:24:10.708623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:10.709239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:10.709239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:10.710492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 737/1488] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:24:10.715758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036f3-e428-4ce5-8457-24d3bee65f87 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:10.716182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036f3-e428-4ce5-8457-24d3bee65f87 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:10.719996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f036f3-e428-4ce5-8457-24d3bee65f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:10.780979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-81f036f3-e428-4ce5-8457-24d3bee65f87 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:24:10.782161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-81f036f3-e428-4ce5-8457-24d3bee65f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:10.833107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:10.876220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 752/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:24:11.165079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:24:11.166630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:11.202372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:11.269125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0775d925-9486-4a89-bb33-c6a88eee1b51 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:24:11.339602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:24:11.340190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:11.340490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:11.340758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:11.342318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:11.421501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:11.449680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:24:11.451096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46a370-780c-4dc9-a905-581e0591b5df service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:24:11.468989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 738/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:24:13.789215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:13.789774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:13.789774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:13.791096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 753/1491] 10.222.0.33 () {54 vars in 927 bytes} [Thu Sep 22 12:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:24:13.796235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab951-47d2-40f0-adf5-0fcad9879fdb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:13.796678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab951-47d2-40f0-adf5-0fcad9879fdb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:13.796809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ab951-47d2-40f0-adf5-0fcad9879fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:13.863852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d73ab951-47d2-40f0-adf5-0fcad9879fdb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:24:13.865002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d73ab951-47d2-40f0-adf5-0fcad9879fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:13.958819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 739/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Thu Sep 22 12:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:24:14.246683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:24:14.248251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:14.348091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bab27c-fa1d-4726-9840-ab2a2deb5460 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:24:14.420776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:24:14.421351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:24:14.421624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:24:14.421886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:24:14.423449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:24:14.431723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:14.528799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:24:14.530146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad0ec10-9e7e-49e6-aa31-754654baec20 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:24:14.548311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 754/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:24:23.957288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=82 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:24:24.546109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=50 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:32.629817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:32.630451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:32.630451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:32.703842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:25:32.705067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:32.799540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 740/1494] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:25:32.807527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:32.808016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:32.808179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:32.883412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:25:32.884559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:32.980649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 755/1495] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:25:32.985442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:32.986888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:32.998285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ceb08-6123-4698-92f7-d65e3d4ec2dc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.058453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:33.092312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.092917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.093198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.093471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.094101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:25:33.094402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.095412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.095412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:25:33.095412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:25:33.096317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.107456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 741/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:25:33.116233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:33.125232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9379cb-6949-4580-b71a-846e2c238677 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.185232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:33.218351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.218950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.219219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.219480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.220097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.221914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.221914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:25:33.221914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:25:33.223084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.278364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 756/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:25:33 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:25:33.282539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:33.284049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:33.317603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e64654-5049-4c48-8ce5-3c0f0d8ca0bc admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.411258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.411911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.412188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.412489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.413132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.414129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.414129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:25:33.414129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:25:33.415309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.457941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 742/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:25:33 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:25:33.461992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:33.463506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:33.523172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ae555-9afb-4bb8-814b-1996eb11f0ea admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.606907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:33.617755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.618352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.618640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.618889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.619514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.620547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.620547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:25:33.620547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:25:33.621417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.633245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 757/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:25:33.637280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:33.638809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:33.649905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee503a-56c3-4905-a096-5f9bc91d61cd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.674553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:33.744164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.744718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.744993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.745260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.745887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.746892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.746892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:25:33.746892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:25:33.747767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcd695-6225-4ad7-91fc-3265182c26f1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.759695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 743/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:25:33.763848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:33.765421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:33.801896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6a180-8f06-4195-bbb5-f5cddc2b06b3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:33.892940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:33.893620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.894398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.894661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] PATH_INFO: `/v3/projects/9131c03327494c689a1dc727cd33c3e3/users/8aba256e346f475d8a27a3d5741622f2/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:33.916612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9131c03327494c689a1dc727cd33c3e3, user_id=8aba256e346f475d8a27a3d5741622f2, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:33.917621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:33.917621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:25:33.917621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:25:33.919526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f137aa-ca2f-463e-8424-3ba44bc92146 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:33.942799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 758/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:25:33 2022] PUT /identity/v3/projects/9131c03327494c689a1dc727cd33c3e3/users/8aba256e346f475d8a27a3d5741622f2/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:25:33.954773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c4dd3-24d9-49c7-accc-3d4a504114c8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:33.955058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c4dd3-24d9-49c7-accc-3d4a504114c8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:33.955329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c4dd3-24d9-49c7-accc-3d4a504114c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.012357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-457c4dd3-24d9-49c7-accc-3d4a504114c8 None None] MFA Rules not processed for user `8aba256e346f475d8a27a3d5741622f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:25:34.013501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-457c4dd3-24d9-49c7-accc-3d4a504114c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:34.089060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 744/1502] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:25:34.102859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.103143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.103414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.104755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 759/1503] 10.222.0.33 () {54 vars in 963 bytes} [Thu Sep 22 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:25:34.109390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c29e3-7a7a-4db1-90d5-222c6e9e53f4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.109672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c29e3-7a7a-4db1-90d5-222c6e9e53f4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.109944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c29e3-7a7a-4db1-90d5-222c6e9e53f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.140704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:34.186046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-895c29e3-7a7a-4db1-90d5-222c6e9e53f4 None None] MFA Rules not processed for user `e10e2b0b66754f128cddc94ca3daf101`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:25:34.187229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-895c29e3-7a7a-4db1-90d5-222c6e9e53f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:34.260564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 745/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Thu Sep 22 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:25:34.268218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:34.270419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:34.319515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:34.373434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05227ea-6cf8-400e-96ad-a91569461bbf None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:34.455530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:34.456168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.457135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.457407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.459084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:34.557629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:34.558939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:34.562365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:34.577675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 760/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:25:34.623528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f176-c53e-40ac-a0d1-6708329fe994 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.623814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f176-c53e-40ac-a0d1-6708329fe994 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.624089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634f176-c53e-40ac-a0d1-6708329fe994 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.625375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 746/1506] 10.222.0.33 () {52 vars in 830 bytes} [Thu Sep 22 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:25:34.629490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:34.630678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:34.726193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea7815-12ea-43fc-a3a4-0f960fe5387f service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:34.817368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:34.817912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.818179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.818440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] PATH_INFO: `/v3/users/8aba256e346f475d8a27a3d5741622f2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:34.830742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aba256e346f475d8a27a3d5741622f2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:34.831779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:34.831779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:25:34.831779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:25:34.832998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:34.842341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:34.898862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aba256e346f475d8a27a3d5741622f2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:34.907698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aba256e346f475d8a27a3d5741622f2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:34.908207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aba256e346f475d8a27a3d5741622f2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:34.922025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aba256e346f475d8a27a3d5741622f2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:34.946728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aba256e346f475d8a27a3d5741622f2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:34.952258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 761/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:25:34 2022] DELETE /identity/v3/users/8aba256e346f475d8a27a3d5741622f2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:25:34.956808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:34.957264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:34.957379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:35.013203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:25:35.014381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:35.088435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:35.102189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 747/1508] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:25:35.111879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:35.113785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:35.212746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf0dcc-727e-422a-b146-9032478aaf17 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:35.269723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:35.290963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:35.291602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:35.291880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:35.292153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:35.293791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:35.400474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:35.401829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f89743-2c4c-4e28-a852-730e73c6b839 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:35.419648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 762/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:25:35.762538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:25:35.763751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:25:35.775608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94e3a0-cf6c-4b96-9ce8-03f5a1a42214 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:25:35.874602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:25:35.875223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:25:35.875496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:25:35.875761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] PATH_INFO: `/v3/projects/9131c03327494c689a1dc727cd33c3e3` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:25:35.886202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9131c03327494c689a1dc727cd33c3e3)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:25:35.887220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:25:35.887220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:25:35.887220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:25:35.888422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:25:35.945652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9131c03327494c689a1dc727cd33c3e3', 'request_id': 'req-aab46a5a-7466-40a9-8ddf-dfa59923c005'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:35.954600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9131c03327494c689a1dc727cd33c3e3', 'request_id': 'req-aab46a5a-7466-40a9-8ddf-dfa59923c005'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:35.961331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9131c03327494c689a1dc727cd33c3e3', 'request_id': 'req-aab46a5a-7466-40a9-8ddf-dfa59923c005'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:35.991468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] The token cache is being invalidate because project 9131c03327494c689a1dc727cd33c3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:25:35.991756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:35.992204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aab46a5a-7466-40a9-8ddf-dfa59923c005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:25:35.993595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 748/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:25:35 2022] DELETE /identity/v3/projects/9131c03327494c689a1dc727cd33c3e3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:25:45.417429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=62 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:25:45.989459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:27:01.688763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:01.689363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:01.689363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:01.690658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 763/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:01.696169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c10fd-f539-4996-9931-7323acce54bf None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:01.696632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c10fd-f539-4996-9931-7323acce54bf None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:01.696939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c10fd-f539-4996-9931-7323acce54bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:01.776904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d72c10fd-f539-4996-9931-7323acce54bf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:01.778097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d72c10fd-f539-4996-9931-7323acce54bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:01.882868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 749/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:02.034881 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:02.037506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:02.153154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01f3be-ce09-4618-94ec-2f1ecd8614b4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:02.158566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:02.159162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:02.159435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:02.159703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:02.168105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:02.169733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3c1c6-9c32-4c4c-bc26-9df66d4ef964 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:02.179724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 764/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:03.265987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f15fc-bc0a-4d33-bf35-860979b3d37b None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:03.266558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f15fc-bc0a-4d33-bf35-860979b3d37b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:03.266558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f15fc-bc0a-4d33-bf35-860979b3d37b None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:03.267877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 750/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:03.274178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1940cc-7519-4ad7-bea6-b44fd9cf9c9d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:03.274642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1940cc-7519-4ad7-bea6-b44fd9cf9c9d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:03.274746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1940cc-7519-4ad7-bea6-b44fd9cf9c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:03.293623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8a1940cc-7519-4ad7-bea6-b44fd9cf9c9d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:03.294828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8a1940cc-7519-4ad7-bea6-b44fd9cf9c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:03.302970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 765/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:27:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:03.454872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f15fc-bc0a-4d33-bf35-860979b3d37b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:03.469027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f15fc-bc0a-4d33-bf35-860979b3d37b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:03.474874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:03.475464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:03.475751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:03.476270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:03.485999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:03.487607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88adffc-2aa2-4fcc-a1a8-e8384a316b53 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:03.497809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 751/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:04.137926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1a39-088a-4352-941f-c22437da25e5 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:04.138133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1a39-088a-4352-941f-c22437da25e5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:04.138391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d1a39-088a-4352-941f-c22437da25e5 None None] PATH_INFO: `/v3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:04.139706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 766/1517] 10.222.0.20 () {54 vars in 925 bytes} [Thu Sep 22 12:27:04 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:27:04.146123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c4144-1b11-4249-a02f-276c19019186 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:04.146409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c4144-1b11-4249-a02f-276c19019186 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:04.146686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c4144-1b11-4249-a02f-276c19019186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:04.204977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-611c4144-1b11-4249-a02f-276c19019186 None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:04.206164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-611c4144-1b11-4249-a02f-276c19019186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:04.222420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:27:04.277835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 752/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 22 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:07.907748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7dda-ff1c-4fed-98f2-a02ce5c110fa None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:07.908182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7dda-ff1c-4fed-98f2-a02ce5c110fa None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:07.908279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7dda-ff1c-4fed-98f2-a02ce5c110fa None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:07.909630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 767/1519] 10.222.0.33 () {54 vars in 961 bytes} [Thu Sep 22 12:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:07.914588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f062b1-921f-4e9f-a6b3-8bb2c823b685 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:07.915033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f062b1-921f-4e9f-a6b3-8bb2c823b685 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:07.915125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f062b1-921f-4e9f-a6b3-8bb2c823b685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:07.935020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-77f062b1-921f-4e9f-a6b3-8bb2c823b685 None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:07.936215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-77f062b1-921f-4e9f-a6b3-8bb2c823b685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:07.945323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 753/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Sep 22 12:27:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:07.952968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce7dda-ff1c-4fed-98f2-a02ce5c110fa None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:07.965817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce7dda-ff1c-4fed-98f2-a02ce5c110fa None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:07.970601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:07.971176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:07.971448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:07.971714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:07.980957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:07.982523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da1fec8-b627-4cbf-86c7-3f8ce91a377c service placement] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:07.993442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 768/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:15.006016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7add3e6-ae94-4f61-bfc9-a98f5cba441e None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:15.006616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7add3e6-ae94-4f61-bfc9-a98f5cba441e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:15.006616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7add3e6-ae94-4f61-bfc9-a98f5cba441e None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:15.007961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 754/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:15.014099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15347d6a-200b-4fa4-8b74-b89131bc4f1c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:15.014564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15347d6a-200b-4fa4-8b74-b89131bc4f1c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:15.014660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15347d6a-200b-4fa4-8b74-b89131bc4f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:15.034229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-15347d6a-200b-4fa4-8b74-b89131bc4f1c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:15.035421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-15347d6a-200b-4fa4-8b74-b89131bc4f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:15.044012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 769/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:15.199930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7add3e6-ae94-4f61-bfc9-a98f5cba441e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:15.215278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7add3e6-ae94-4f61-bfc9-a98f5cba441e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:15.220622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:15.221208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:15.221488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:15.221762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:15.230499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:15.232060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ad711-cbd3-4979-af72-47ee5e4affb9 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:15.241508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 755/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:16.312430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387f8ee-8969-4787-8d84-2900433d5312 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:16.313012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387f8ee-8969-4787-8d84-2900433d5312 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:16.313012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2387f8ee-8969-4787-8d84-2900433d5312 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:16.314299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 770/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:16.320214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96636285-08c2-40ff-8f37-73bc7f4e5050 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:16.320634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96636285-08c2-40ff-8f37-73bc7f4e5050 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:16.320808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96636285-08c2-40ff-8f37-73bc7f4e5050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:16.340382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-96636285-08c2-40ff-8f37-73bc7f4e5050 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:16.341575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-96636285-08c2-40ff-8f37-73bc7f4e5050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:16.349650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 756/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 12:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:16.501206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387f8ee-8969-4787-8d84-2900433d5312 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:16.515347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2387f8ee-8969-4787-8d84-2900433d5312 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:16.521089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:16.521679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:16.521972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:16.522753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:16.530779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:16.532399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87952254-14ff-4a75-9d07-7a586d06207c service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:16.542388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 771/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:20.423811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:20.424421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:20.424421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:20.425727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 757/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:20.430867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137104ad-4776-4ecf-ab82-8aa930c288e4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:20.431264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137104ad-4776-4ecf-ab82-8aa930c288e4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:20.431412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137104ad-4776-4ecf-ab82-8aa930c288e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:20.450254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-137104ad-4776-4ecf-ab82-8aa930c288e4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:20.451435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-137104ad-4776-4ecf-ab82-8aa930c288e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:20.459139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 772/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:20.749649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:20.751340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:20.828859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3d542-0703-4184-8abc-9c01780cfefe None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:20.832354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:20.832938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:20.833211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:20.833477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:20.842688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:20.844091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8d110-3417-4417-ba32-17d18ee479bf service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:20.853433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 758/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:22.975906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbed34-adc4-4863-91c4-9726508d9f62 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:22.976769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbed34-adc4-4863-91c4-9726508d9f62 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:22.976769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbed34-adc4-4863-91c4-9726508d9f62 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:22.978053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 773/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:22.983188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451043a-0d89-4c41-83bb-a2471b3eecbe None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:22.983590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451043a-0d89-4c41-83bb-a2471b3eecbe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:22.983732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451043a-0d89-4c41-83bb-a2471b3eecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:23.003446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4451043a-0d89-4c41-83bb-a2471b3eecbe None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:23.004686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4451043a-0d89-4c41-83bb-a2471b3eecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:23.013000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 759/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:23.300998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbed34-adc4-4863-91c4-9726508d9f62 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:23.312077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbed34-adc4-4863-91c4-9726508d9f62 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:23.315578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:23.316164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:23.316481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:23.316750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:23.325919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:23.327333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80905cb-0068-4ffd-b7ab-0d1799ce52db service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:23.336534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 774/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:26.873873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:26.873873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:26.873873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:26.875190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 760/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:26.880423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26fff1-3615-42cf-9c98-c3f2db664ded None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:26.880812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26fff1-3615-42cf-9c98-c3f2db664ded None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:26.880962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26fff1-3615-42cf-9c98-c3f2db664ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:26.888488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:27:26.900889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ce26fff1-3615-42cf-9c98-c3f2db664ded None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:26.902071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ce26fff1-3615-42cf-9c98-c3f2db664ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:26.909763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 775/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:27.211330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:27.212983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:27.276920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:27:27.290685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe18f1-3009-4f06-bec0-0e407c7b95d5 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:27.294260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:27.294847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:27.295126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:27.295384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:27.304102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:27.305554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bdb229-9e8e-47ed-8e15-3c76909d0fc6 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:27.314742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 761/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:29.483771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:29.484412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:29.484412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:29.485665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 776/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 12:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:27:29.491331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbb808-89a1-4f48-8cf7-35e37941f5d6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:29.491737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbb808-89a1-4f48-8cf7-35e37941f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:29.491866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbb808-89a1-4f48-8cf7-35e37941f5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:29.511695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a2bbb808-89a1-4f48-8cf7-35e37941f5d6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:27:29.512890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a2bbb808-89a1-4f48-8cf7-35e37941f5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:29.520777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 762/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 12:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:27:29.831537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:27:29.833510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:27:29.847523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70e9df-7d81-450a-bcf4-5c150f2d85e7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:27:29.852946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:27:29.853458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:27:29.853767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:27:29.854056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:27:29.863526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:27:29.865055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c584cb0-6c38-45a0-9b5d-19ef5eb6aefa service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:27:29.876400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 777/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:27:39.506778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=11 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:27:39.872505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=8 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:54.736490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af7b61-d890-4f29-984b-fc24e8896eeb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.738198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af7b61-d890-4f29-984b-fc24e8896eeb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.738198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af7b61-d890-4f29-984b-fc24e8896eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.767640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418aa4a-934b-4969-85a6-5fd8ec792bfd None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.768269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-01af7b61-d890-4f29-984b-fc24e8896eeb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:54.768927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418aa4a-934b-4969-85a6-5fd8ec792bfd None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.769261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-01af7b61-d890-4f29-984b-fc24e8896eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:54.769707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418aa4a-934b-4969-85a6-5fd8ec792bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.787896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 763/1540] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:54.800332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.800332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.800332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.812236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1418aa4a-934b-4969-85a6-5fd8ec792bfd None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:54.813834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1418aa4a-934b-4969-85a6-5fd8ec792bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:54.829313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:54.830120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 778/1541] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:28:54.830504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:54.840354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.841038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.841537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.847084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 764/1542] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:28:54.852657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:54.870343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc72b6-92f9-4000-b24a-899f8b69526e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:54.889286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:54.890833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:54.893718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:54.894348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.894627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.895436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.896080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:54.896405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:54.898162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:54.911756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 779/1543] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:28:54.918990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:54.927490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 765/1544] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:54.929425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82dc4a5-0947-4abb-83be-fa22d710ec9e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:54.935958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:54.959966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:54.961406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.963797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.964404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.965756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:54.966372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:54.969431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7fa667-f389-41bd-a688-9f2ed6a1cd6e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:54.977103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:28:54.977103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:28:54.977103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:28:54.978513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:54.998079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:54.998079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:54.998378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:54.998744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:54.999153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:54.999904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 780/1545] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:28:55.001738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:28:55.001738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:28:55.001738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:28:55.001738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f8979c-2473-4b42-88e3-4af71fef61f5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.003266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.019084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b523176-7fc9-42b8-a173-ca703073a4e0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.027445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.028489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.029062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.029567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.030618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.033500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ce990c-48d7-4b9c-bc77-edfd9444fe43 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.083828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 781/1546] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 393 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.090336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6844-c5ee-4d5f-ab12-bb702d9ff9c8 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.090840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6844-c5ee-4d5f-ab12-bb702d9ff9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.091112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c6844-c5ee-4d5f-ab12-bb702d9ff9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.100013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 766/1547] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:54 2022] POST /identity/v3/projects => generated 357 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.102009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.102288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.102556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.122138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fd2c6844-c5ee-4d5f-ab12-bb702d9ff9c8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.123837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fd2c6844-c5ee-4d5f-ab12-bb702d9ff9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.138296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.139536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.197665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 782/1548] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.198878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.199157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.199431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.219717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.220061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 767/1549] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.220964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.221088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.229435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 783/1550] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.230196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.231332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e659dd-aa1a-4f49-bc30-141e7326d6df None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.238108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.238827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.239102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.239374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.239995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.240114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea67a0-93d3-4eb1-9d34-166e21e061e9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.241936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573558ab-bb3d-4133-83a3-1a5881078458 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.244902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.245517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.245792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.246058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.246668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.247672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:28:55.247672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:28:55.247672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:28:55.248939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71c302-d550-4737-a6b2-9c125d620178 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.265423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 768/1551] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/users => generated 297 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.267045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190e83-c300-41f9-af45-3c9c34b00215 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.267630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190e83-c300-41f9-af45-3c9c34b00215 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.267906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190e83-c300-41f9-af45-3c9c34b00215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.273183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 784/1552] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/users => generated 279 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.274197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472485a-eb10-4f90-8b03-2f70e2b6485f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.274483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472485a-eb10-4f90-8b03-2f70e2b6485f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.274753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472485a-eb10-4f90-8b03-2f70e2b6485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.292079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ae190e83-c300-41f9-af45-3c9c34b00215 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.293376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ae190e83-c300-41f9-af45-3c9c34b00215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.294232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6472485a-eb10-4f90-8b03-2f70e2b6485f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.295384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6472485a-eb10-4f90-8b03-2f70e2b6485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.304430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 769/1553] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.304430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591052-1d79-48e8-8805-3df4066328fc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.304430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591052-1d79-48e8-8805-3df4066328fc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.304430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2591052-1d79-48e8-8805-3df4066328fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.306248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 785/1554] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.307588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9aea-a7f7-4586-bcd8-5245685382ef None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.307865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9aea-a7f7-4586-bcd8-5245685382ef None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.308134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9aea-a7f7-4586-bcd8-5245685382ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.327825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-134c9aea-a7f7-4586-bcd8-5245685382ef None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.329032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d2591052-1d79-48e8-8805-3df4066328fc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.329174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-134c9aea-a7f7-4586-bcd8-5245685382ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.330194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d2591052-1d79-48e8-8805-3df4066328fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.337218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 786/1555] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.338396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.338524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 770/1556] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.338726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.338947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.339564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.339873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.340142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.360822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.361018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.362014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.362151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.370491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 771/1558] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.370633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 787/1558] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.371271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.371391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.380784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b42ee-3282-45ac-9d51-02ec67b2e37f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.381078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca37a313-dd55-407c-ad23-efe0bd010ce2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.385323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.385524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.385915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.386071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.386186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.386350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.386464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.386623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.387058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.387225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.388677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0eacf1-c8d2-4b71-97a9-0c60a8ad6660 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.388995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355c16f-70c3-450b-b3dd-43b7129b8518 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.398839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 772/1559] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:55.399244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 788/1560] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:55.400100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.400209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad431f-9711-448d-8823-173d5dc2c0db None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.400405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.400541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad431f-9711-448d-8823-173d5dc2c0db None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.400671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.400777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad431f-9711-448d-8823-173d5dc2c0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.419869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.420562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-29ad431f-9711-448d-8823-173d5dc2c0db None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.421080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.421720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-29ad431f-9711-448d-8823-173d5dc2c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.429287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 773/1561] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.430232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 789/1562] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.430354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.431216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.431494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.431764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.442605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3df889-1302-43fd-a3e4-dfcaef0a1b42 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.447197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.447787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.448067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.448365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.449028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.449328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.450846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:55.450966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.452040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:55.460744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 790/1563] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:55.461393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.466362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 774/1564] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.467159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.470629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecaf7ba-9f29-4413-a17b-14fbe9693566 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.476446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.477284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.477563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.477681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575e85a-4d73-4f5a-9214-eec51cf23d3d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.477830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.478462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.478759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.480366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.482371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.482941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.483220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.483493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.484342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.484652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.486216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.493172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 791/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.494119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.503524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a127bd-1bde-4c0f-bdb2-ae45c7f6fb89 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.505381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 775/1566] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.506236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.508065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.508671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.508947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.509212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.509832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.511558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.516412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac593f-c0a7-43ea-bb60-f4c141f10d64 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.521696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.522271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.522543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.522815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.523012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 792/1567] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:55.523562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.523849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.525168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.533451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d0a08-3a15-4239-a226-640d8d571d18 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.535226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 776/1568] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:55.536542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.538693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.539262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.539539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.539811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.540462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.540760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.542326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.547937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a959bf-7145-4e8e-9891-2804164a2b8f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.552531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.553105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.553381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.553656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.553787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 793/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.554279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.554503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.554607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.556124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.562782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:55.569201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 777/1570] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.570182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.570873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835fcad-c443-46cb-abd9-d5479d6ee8cb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.575754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.576561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.576836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.577103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.577729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:28:55.578025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.579597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.580794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b7861b-d410-4bbd-9090-6e673f59272e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.581166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:55.586466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.587091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.587329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.587608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.588246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.590351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.595345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 794/1571] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:28:55.596444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.606562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd7740-4c15-478d-88ce-9fb889e909f8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.611424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.611992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.612378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.612578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.613185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.615241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.625406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 778/1572] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.626244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.649225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 795/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.649225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.690854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753018-b4da-421a-b713-57b0914f06eb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.707973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7148d62-6062-41c2-9b0a-cceb1f34b5e7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.713113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.713773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.714042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.714324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] PATH_INFO: `/v3/projects/763a056f453c4b9e8c649907c9f0b4ce/users/368a8b674a824ace996f84d0454444da/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.730409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763a056f453c4b9e8c649907c9f0b4ce, user_id=368a8b674a824ace996f84d0454444da, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.733141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.746585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 796/1574] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:55 2022] PUT /identity/v3/projects/763a056f453c4b9e8c649907c9f0b4ce/users/368a8b674a824ace996f84d0454444da/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:55.747100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.747651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.748283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.748557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.748828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.749457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.751341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.778586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 779/1575] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 405 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.779487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.833331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9760052-70db-4f03-8e8e-a5d74d45e330 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.838146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.838934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.839218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.839566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] PATH_INFO: `/v3/projects/fb62ca26c4fe47deb0a21511a96e6403/users/0847f00afbde44068fcf1cedbce24be5/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.840989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50573e94-873c-4b07-b5e7-16d0a608056a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.845547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.846139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.846413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.846680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.847288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.849744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.855115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb62ca26c4fe47deb0a21511a96e6403, user_id=0847f00afbde44068fcf1cedbce24be5, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.857811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.864324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:55.873437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 797/1576] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:55 2022] PUT /identity/v3/projects/fb62ca26c4fe47deb0a21511a96e6403/users/0847f00afbde44068fcf1cedbce24be5/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:55.874232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.879599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 780/1577] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.880609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.918399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:55.940646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee42c87-b549-47ac-aa86-c502518ead53 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.942693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531660a7-355f-4d50-8fce-f50ce8bc6158 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:55.946299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.946957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.947233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.947387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:55.947513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.947983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:55.948264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:55.948396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.948571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:55.949189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:55.950211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.951086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:55.980873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 798/1578] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 405 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.980873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:55.981174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 781/1579] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:55.981967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.041541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b8009-2315-4146-95dc-48b011793d0a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.043989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ff1b5-5b8e-47ba-9fb0-e52370393cd8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.046271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.046929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.047207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.047473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.048089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.048666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.049304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.049583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.049858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.050028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d29d95-942a-4293-8ec9-44792c3abccb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.050476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.052970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.074082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 799/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.075371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.075659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.079040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.079203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 782/1581] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:55 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.079203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.091440 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5277d-76dd-455c-ba99-e24f25ab739c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.099671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.100460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.100747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.101021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.101935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.103856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c630c0b-7537-4a4c-b813-59c15cd26a58 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.106357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] MFA Rules not processed for user `368a8b674a824ace996f84d0454444da`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:56.107572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:56.127004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 783/1582] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/users => generated 303 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.128543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.128974 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.129414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.149898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:56.151100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:56.158740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 784/1583] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:56.159488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.169198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08454b1f-2813-4fa0-8b29-75973563f9ee None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.173790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.174373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.174648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.174912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 800/1584] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:56.178947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.178947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.179109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.179191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.186510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027b536-fe2a-4221-8c86-75a4096b95de None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.192614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.193433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.193714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.194270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.195221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.197200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.199721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 785/1585] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.201245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.211854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f207e-54d5-4344-870e-bdb50e928970 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.216680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.217273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.217552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.217820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.218443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.218582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 801/1586] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.219300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.220383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.229644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675aef46-e8f0-4af6-a4ff-2478d5fa7326 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.235713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.236357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.236640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.237034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.237729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.239953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.246580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 786/1587] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.246580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.250912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 802/1588] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.254740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.255155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b59c471-b9dc-4b08-bb32-70f932c8e895 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.260034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.260647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.260921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.261184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.261808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.263379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.264399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a511b-baa2-46ca-ab47-1defcd60c228 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.265871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=14,keystone:SELECT=86 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:56.269329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.270430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.270711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.270974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.271600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.273199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.280031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 787/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.280031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.288861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 803/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.288861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.291831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:56.299440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee120969-8594-400c-8240-b559fc2b7f47 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.305410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.306071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.306418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.307224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.307366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.309310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.339602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 804/1591] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.339602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.339905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89364f-0837-4813-ae53-d3d869d525e9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.387905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.388535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.388811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.389107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.398385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.398924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86075bb-77c2-4a6f-8e12-8a62c9be647a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.399801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.403820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.404580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.404862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.405134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.405801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.407418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.409159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 788/1592] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:56.409924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.418421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 805/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.419264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.424553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61c419-8c50-4d6f-93f5-ab103ae20a17 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.429808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058cdc6-e970-411d-82a9-ca7b508f4e67 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.430555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.431196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.431462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.431731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.432412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.434151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.436957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.438018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.438295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.438673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.439305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.441399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.450818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 789/1594] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.450818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.454665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 806/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.454665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.460015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6748828-1435-4191-b669-3f29148576c1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.463469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adaebf-ae1b-4277-b22a-0c1af3ea6d3a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.465036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.465702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.466040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.466464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.467069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.468837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.468984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.469245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.469594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.469935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.471112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.472462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.482408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 790/1596] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.483391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.489266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 807/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.490163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.495146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0b239-35e6-4336-b9fd-852041b83feb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.500104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.500476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51907b-9df4-4cfb-9ed2-bb13c0a2c664 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.501148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.501434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.501733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.502361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.503951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.506149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.506866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.507245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.507637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.508387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.510504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.516669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 791/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.517406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.529027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7d142c-ce5a-4c33-affe-bf0c4d4a8c26 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.534577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.535169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.535492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.536263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.539949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.540090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 808/1599] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/users => generated 285 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:56.540090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.540264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.551415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167675e2-953b-464e-a344-d8b3aab2139b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.553304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 792/1600] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.554212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.558352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.559082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.560207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.560627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.562014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.564040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.565287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099068-308a-40cc-bc49-96adc7def2d4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.571616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.572378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.572995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.573322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.573952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.575600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.576334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 809/1601] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.577261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.589993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352683f8-26de-46c4-a9b9-4f4948072f34 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.593706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 793/1602] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.593706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.595281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.595942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.596248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.596690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] PATH_INFO: `/v3/projects/89fe74d1d646452e83cb93420766c0a2/users/49c772b0c2bb49a4ab4048d980bd20a8/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.607267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7102b-763f-4222-8902-3120cf970801 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.617506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=89fe74d1d646452e83cb93420766c0a2, user_id=49c772b0c2bb49a4ab4048d980bd20a8, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.620638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.622120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.622917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.623395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.623522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] PATH_INFO: `/v3/projects/acfd00e7112b409bba3cdf9480508b02/users/e125a85011fe402ab2d5782d8fd39dfc/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.636803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 810/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/89fe74d1d646452e83cb93420766c0a2/users/49c772b0c2bb49a4ab4048d980bd20a8/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:56.637733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.642928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfd00e7112b409bba3cdf9480508b02, user_id=e125a85011fe402ab2d5782d8fd39dfc, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.644146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:28:56.644146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:28:56.644146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:28:56.646966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.665012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 794/1604] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/acfd00e7112b409bba3cdf9480508b02/users/e125a85011fe402ab2d5782d8fd39dfc/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:56.666005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.714477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:56.715942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f552eb-bd29-4a1f-9992-96b713d29e7a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.722680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:56.745875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb66ec-517f-4aa9-a9ba-2f968ddbac9f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.752769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.753728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.754019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.754827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.756313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.757878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.773428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 795/1605] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:56.773428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.784754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1c1ce-1622-4d54-be57-aca4e2a1f2e7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.791865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.792570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.792849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.793131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] PATH_INFO: `/v3/projects/40ddbbe59b90403fb3ea9212d10727aa/users/55a7f7ba0b27479f8d1e68d808637d55/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.795742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.796380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.796663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.796934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] PATH_INFO: `/v3/projects/c3d5c01356eb45f9b87fddc5526b6de1/users/eea4148a487046a384ffab5d7d698b54/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.814812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3d5c01356eb45f9b87fddc5526b6de1, user_id=eea4148a487046a384ffab5d7d698b54, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.818111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.821858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ddbbe59b90403fb3ea9212d10727aa, user_id=55a7f7ba0b27479f8d1e68d808637d55, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.825505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.837424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 811/1606] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/c3d5c01356eb45f9b87fddc5526b6de1/users/eea4148a487046a384ffab5d7d698b54/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:56.838931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.846813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 796/1607] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/40ddbbe59b90403fb3ea9212d10727aa/users/55a7f7ba0b27479f8d1e68d808637d55/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:56.847647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:56.911212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04af201-4bde-4f63-b58a-43b77e53f6d8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.925231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59449fc0-d408-4c98-a5a1-ff87cab5953c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:56.930291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.931384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.931761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.932179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] PATH_INFO: `/v3/projects/e47fbc746f4c4c3c8a1adfd454035fce/users/78ae9d0f1ba04d10ac840829b4236370/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.948141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47fbc746f4c4c3c8a1adfd454035fce, user_id=78ae9d0f1ba04d10ac840829b4236370, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.951502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6450da9b-eddb-49aa-90f7-fb05bf6cc288 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:56.966062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 797/1608] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/e47fbc746f4c4c3c8a1adfd454035fce/users/78ae9d0f1ba04d10ac840829b4236370/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:56.975264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:56.975944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.976224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.976809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] PATH_INFO: `/v3/projects/ff03a6fd472a433eaa40ebf6351668ed/users/7fac2e1e4e34482aab46f7c4b9039a82/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.979163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:56.979743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:56.980727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:56.993769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff03a6fd472a433eaa40ebf6351668ed, user_id=7fac2e1e4e34482aab46f7c4b9039a82, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:56.996751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5383b7-025b-4e3e-970d-185f098a2d08 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.000148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.001471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.013324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 812/1609] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:56 2022] PUT /identity/v3/projects/ff03a6fd472a433eaa40ebf6351668ed/users/7fac2e1e4e34482aab46f7c4b9039a82/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:57.015144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5677cff-7b9f-423b-b802-46b1e76ccf53 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.015440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5677cff-7b9f-423b-b802-46b1e76ccf53 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.015705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5677cff-7b9f-423b-b802-46b1e76ccf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.045351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c5677cff-7b9f-423b-b802-46b1e76ccf53 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.046728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c5677cff-7b9f-423b-b802-46b1e76ccf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.067524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:57.074168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 798/1610] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.074915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.084721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df13aaa-0818-4c01-8bef-90fbd39f3829 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.116225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:57.121379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 813/1611] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.123264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bcfc-82a6-4465-b78f-2ce64e7e34d2 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.123709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bcfc-82a6-4465-b78f-2ce64e7e34d2 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.124489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766bcfc-82a6-4465-b78f-2ce64e7e34d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.147270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3766bcfc-82a6-4465-b78f-2ce64e7e34d2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.148397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3766bcfc-82a6-4465-b78f-2ce64e7e34d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.155172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.155172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.155172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.155172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] PATH_INFO: `/v3/projects/836bc3b6063446b4ae19121da3539b06/users/1926018aaeed478f953b541c37a43885/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.162894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 814/1612] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.168469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e85f7-54b3-435f-b65e-8eb47188b558 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.168469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e85f7-54b3-435f-b65e-8eb47188b558 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.168706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e85f7-54b3-435f-b65e-8eb47188b558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.173457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836bc3b6063446b4ae19121da3539b06, user_id=1926018aaeed478f953b541c37a43885, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.176887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb08090-247f-4bfe-976b-d7aab3017415 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.190340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2c6e85f7-54b3-435f-b65e-8eb47188b558 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.191587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2c6e85f7-54b3-435f-b65e-8eb47188b558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.195187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 799/1613] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:57 2022] PUT /identity/v3/projects/836bc3b6063446b4ae19121da3539b06/users/1926018aaeed478f953b541c37a43885/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:57.196714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.197068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.197781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.242735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] MFA Rules not processed for user `78ae9d0f1ba04d10ac840829b4236370`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.244657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.265179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 815/1614] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.266903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.280045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.280350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.319487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] MFA Rules not processed for user `7fac2e1e4e34482aab46f7c4b9039a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.321226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.329246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 800/1615] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.330435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.411193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 816/1616] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.412064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.431464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6fcd8-94fa-400f-966c-578799ed01a4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.435884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.436545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.436826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.437152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.447775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.449302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.460371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 801/1617] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:57.461388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.471132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c25c-b17d-46f8-b45f-fc1eb5378743 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.475150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.475762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.476043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.476337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.477907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:57.486691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34209145-0d13-4d5e-9393-1d7a471fc0d0 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.490704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.491337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.491614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.491887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.492051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.493681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.507287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.508859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 802/1618] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:57.509409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.510081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.521122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45c36c-8593-4678-a951-cfc51c025f9c service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.521316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 817/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:57.522291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.524043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.526908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.527735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.528235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.528626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.529511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.532190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eedc7-118b-4ba7-9caf-5dbfe5963fb4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.539417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a917354-4d0a-4d05-868d-27301b342e78 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.544574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.545250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.545533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.545806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.546671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 803/1620] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:57.552537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4484286-011e-471c-8b91-dcde4de9fd13 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.552736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4484286-011e-471c-8b91-dcde4de9fd13 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.553112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4484286-011e-471c-8b91-dcde4de9fd13 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.555077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 804/1621] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 22 12:28:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:28:57.562384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4484286-011e-471c-8b91-dcde4de9fd13 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.565534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.568648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.580451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4484286-011e-471c-8b91-dcde4de9fd13 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.584350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 818/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:57.585443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.587367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.588269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.588736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.589248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.590433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.594853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc3e9e0-a8b0-4f16-92e1-23c15910745d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.601793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcb31c-314c-44f0-b9b5-df1922a7d5cf service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.607478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.608155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.608830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.609297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] PATH_INFO: `/v3/projects/836bc3b6063446b4ae19121da3539b06/users/1926018aaeed478f953b541c37a43885/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.611460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=836bc3b6063446b4ae19121da3539b06, user_id=1926018aaeed478f953b541c37a43885, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.614797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d969b5-cf7f-44a4-bf77-db13aa678429 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.637904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 819/1623] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:57 2022] PUT /identity/v3/projects/836bc3b6063446b4ae19121da3539b06/users/1926018aaeed478f953b541c37a43885/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:57.637904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.637904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.637904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.647859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 805/1624] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/projects => generated 403 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:57.650201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.650723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.651416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.659320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:57.694473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] MFA Rules not processed for user `1926018aaeed478f953b541c37a43885`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.696597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.698782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:57.700336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:57.821128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 806/1625] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.822633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.824627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 820/1626] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:57.834800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.872449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e43c7-146c-42ab-80fe-7dca33defcd4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.883519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.883981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.884372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.884644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.939916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73065c4-2251-465c-b4b0-a554a029de8a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:57.946950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:57.947936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:57.948433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:57.948928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:57.950765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.954139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.965538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:57.967996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:57.981287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 821/1627] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:57.982344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:57.987721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 807/1628] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:28:57 2022] POST /identity/v3/users => generated 308 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:28:57.989114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.019557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fd4bf-3b52-4262-b9d1-d0e795578aff admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.029139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.034706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.035284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.035793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.036684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.038502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.045548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:58.059335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 808/1629] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:58.062100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.065382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ee7ab-52f4-4de1-a57b-efc109ba2605 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.072847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.073472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.073783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.074063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.074713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fdf61-8263-4594-874c-81ca6aa61840 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.079994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.081857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.082512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.082894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] PATH_INFO: `/v3/projects/b9bffc6bed2b453a969094d233f1aa44/users/d4491aab55964f309a2de95a603494b8/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.090301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.091929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.103934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bffc6bed2b453a969094d233f1aa44, user_id=d4491aab55964f309a2de95a603494b8, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.106812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.108546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 822/1630] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:58.124274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 809/1631] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:58 2022] PUT /identity/v3/projects/b9bffc6bed2b453a969094d233f1aa44/users/d4491aab55964f309a2de95a603494b8/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:58.128246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.143822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1167c-4278-4e33-8800-92648421fe05 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.227466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.229313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.229736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.231725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.232719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.235743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.260353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.268662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 823/1632] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:28:58.278116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5fd04e-962d-4a7b-9534-9ffd7adfda29 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.288256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.289708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.290120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.290705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] PATH_INFO: `/v3/projects/b9bffc6bed2b453a969094d233f1aa44/users/d4491aab55964f309a2de95a603494b8/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.292870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9bffc6bed2b453a969094d233f1aa44, user_id=d4491aab55964f309a2de95a603494b8, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.296751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.297475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471dffd-fa83-4d1b-b96b-232250962d1c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.342180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 810/1633] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:28:58 2022] PUT /identity/v3/projects/b9bffc6bed2b453a969094d233f1aa44/users/d4491aab55964f309a2de95a603494b8/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:28:58.345063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6006c-267a-4d32-9207-6f1116807728 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.345329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6006c-267a-4d32-9207-6f1116807728 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.350102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6006c-267a-4d32-9207-6f1116807728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.412858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0ef6006c-267a-4d32-9207-6f1116807728 None None] MFA Rules not processed for user `d4491aab55964f309a2de95a603494b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:58.413723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0ef6006c-267a-4d32-9207-6f1116807728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:58.438712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:58.443932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1674d-1acc-4e9b-a679-1ad8fb174cd4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.580521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 811/1634] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:58 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:58.584409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de40c3-25d0-4f0a-8090-013d61fc669a None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.584799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de40c3-25d0-4f0a-8090-013d61fc669a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.585192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de40c3-25d0-4f0a-8090-013d61fc669a None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.590658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 812/1635] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 22 12:28:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:28:58.591754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de40c3-25d0-4f0a-8090-013d61fc669a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:58.594465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.595116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.595385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.595713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.623391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de40c3-25d0-4f0a-8090-013d61fc669a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:58.629616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:58.630276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.630555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.630839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.644266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.646903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad9935-c9f8-4b0e-88e7-8aa32598736a service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.663882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 813/1636] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:58.666936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:58.668574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:58.669031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:58.682703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:58.701440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:58.703003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:58.757799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:58.757799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:58.758251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 824/1637] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:58.959525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 814/1638] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:28:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:58.989830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:59.003208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389c269-4e1a-40be-90ca-29a1755a9f8e service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:59.008670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:59.010338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:59.010959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:59.012890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:59.041507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:59.042981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ae63e-581c-4608-b2ee-fe4ff51a0e95 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:59.055338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 825/1639] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:28:59.278996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:59.448982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15184198-854a-4352-aa3d-bbff160ae124 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:28:59.455911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:28:59.456601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:59.456888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:59.457203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] PATH_INFO: `/v3/users/7fac2e1e4e34482aab46f7c4b9039a82` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:59.459112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fac2e1e4e34482aab46f7c4b9039a82)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:28:59.460551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: WARNING py.warnings [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:28:59.460551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: warnings.warn(msg) Sep 22 12:28:59.460551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: Sep 22 12:28:59.461947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:28:59.567779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:28:59.597583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fac2e1e4e34482aab46f7c4b9039a82'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:28:59.616778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fac2e1e4e34482aab46f7c4b9039a82'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:28:59.658823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fac2e1e4e34482aab46f7c4b9039a82'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:28:59.659805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fac2e1e4e34482aab46f7c4b9039a82'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:28:59.672409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fac2e1e4e34482aab46f7c4b9039a82'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:28:59.680488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 815/1640] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:28:59 2022] DELETE /identity/v3/users/7fac2e1e4e34482aab46f7c4b9039a82 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:28:59.684715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9cf9-8435-413f-a749-23dcdbbd2e39 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:28:59.685617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9cf9-8435-413f-a749-23dcdbbd2e39 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:28:59.685991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba9cf9-8435-413f-a749-23dcdbbd2e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:28:59.719196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6bba9cf9-8435-413f-a749-23dcdbbd2e39 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:28:59.720654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6bba9cf9-8435-413f-a749-23dcdbbd2e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:28:59.886122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 826/1641] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:28:59.917551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:28:59.919290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:00.048130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d7565-9ace-40ef-a624-20023830ab1b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:00.054820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:00.056910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.057410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.057894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.072032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:00.076956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69091d9-d7ab-4223-9181-c46997ec96a0 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:00.092939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 816/1642] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:00.444132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.444712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.444712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.515138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] MFA Rules not processed for user `eea4148a487046a384ffab5d7d698b54`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:00.524273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:00.528723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3912eae-bfd8-492e-b36a-d2cb52368416 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.529261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3912eae-bfd8-492e-b36a-d2cb52368416 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.530115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3912eae-bfd8-492e-b36a-d2cb52368416 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.533077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 817/1643] 10.222.0.33 () {54 vars in 965 bytes} [Thu Sep 22 12:29:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:00.542403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d479-2141-4544-8a12-35718f995301 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.543020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d479-2141-4544-8a12-35718f995301 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.543377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d479-2141-4544-8a12-35718f995301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.614210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-21f4d479-2141-4544-8a12-35718f995301 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:00.615483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-21f4d479-2141-4544-8a12-35718f995301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:00.705681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 827/1644] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:00.711722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:00.715123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:00.770865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d07491-3293-46f0-8f60-edb2ba73109e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:00.782215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:00.784703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.785156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.787400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.788217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:00.792270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:00.824345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 818/1645] 10.222.0.33 () {58 vars in 1062 bytes} [Thu Sep 22 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:00.832037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff0ce-10fe-4bed-86b3-56ff250b6ffe None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.833100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff0ce-10fe-4bed-86b3-56ff250b6ffe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.833207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdff0ce-10fe-4bed-86b3-56ff250b6ffe None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.834615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 819/1646] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 22 12:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:29:00.845837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:00.846598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:00.847078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:00.891308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 828/1647] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:00 2022] POST /identity/v3/projects => generated 401 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:00.895735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:00.912904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:00.913988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:00.965887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:01.005138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:01.006046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d35bb9-8f30-4eb2-8ee6-ee9ca4cbc8a8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.013283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.015440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.015961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.018436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.021539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.026320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.054037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 820/1648] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:29:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:01.068099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 829/1649] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:00 2022] POST /identity/v3/users => generated 307 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:01.069133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.072390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.072912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:01.086074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0de2c-f9bb-49ac-be04-7998feca90e0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.093152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.144372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 830/1650] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:01.146747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.180584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d6752c-8ca8-4766-acbb-15e391cf1b29 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.199671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.200799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.202470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.202866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] PATH_INFO: `/v3/projects/cd403ec87b894c1cbf6adf020919ac66/users/d0cef7a208fb403db0c8f590b8e8d7a0/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.247198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd403ec87b894c1cbf6adf020919ac66, user_id=d0cef7a208fb403db0c8f590b8e8d7a0, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.257763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.258044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f961389-c7d2-4c8b-be1c-cc5dde412da9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.266296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.267557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.267959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.268363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.287106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 831/1651] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:01 2022] PUT /identity/v3/projects/cd403ec87b894c1cbf6adf020919ac66/users/d0cef7a208fb403db0c8f590b8e8d7a0/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:01.288758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.300837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.304527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.338245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 821/1652] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:01.420751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.422938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:01.441173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e051ef-bae4-48da-92e3-b76e672a12f8 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.471538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2ae85-5818-46f4-9c24-6f32a92b44ee admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.477966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.478222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.478509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.478778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.479422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.481068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.512782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 832/1653] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:01 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:01.513720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.532207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6735b91-5c5e-46da-8474-f4590def5c81 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:01.538231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.539022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.539309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.539581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] PATH_INFO: `/v3/projects/cd403ec87b894c1cbf6adf020919ac66/users/d0cef7a208fb403db0c8f590b8e8d7a0/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.541601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd403ec87b894c1cbf6adf020919ac66, user_id=d0cef7a208fb403db0c8f590b8e8d7a0, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.543430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:01.544278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.545892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:01.546311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:01.547183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] PATH_INFO: `/v3/projects/ff03a6fd472a433eaa40ebf6351668ed` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:01.560609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff03a6fd472a433eaa40ebf6351668ed)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:01.571047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:01.578705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 833/1654] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:01 2022] PUT /identity/v3/projects/cd403ec87b894c1cbf6adf020919ac66/users/d0cef7a208fb403db0c8f590b8e8d7a0/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:01.718167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff03a6fd472a433eaa40ebf6351668ed', 'request_id': 'req-d50dc677-197e-4c5f-984a-d7d42393aa5b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:01.741074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff03a6fd472a433eaa40ebf6351668ed', 'request_id': 'req-d50dc677-197e-4c5f-984a-d7d42393aa5b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:01.771867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff03a6fd472a433eaa40ebf6351668ed', 'request_id': 'req-d50dc677-197e-4c5f-984a-d7d42393aa5b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:01.816342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] The token cache is being invalidate because project ff03a6fd472a433eaa40ebf6351668ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:01.818476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:01.826846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:01.832455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 822/1655] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:01 2022] DELETE /identity/v3/projects/ff03a6fd472a433eaa40ebf6351668ed => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:01.836004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:01.838020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:02.017907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19279f-fa7b-40d6-befe-6aacde737221 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:02.114628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:02.197197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:02.198025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:02.198400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:02.198766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] PATH_INFO: `/v3/users/d4491aab55964f309a2de95a603494b8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:02.200109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4491aab55964f309a2de95a603494b8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:02.202209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:02.305768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4491aab55964f309a2de95a603494b8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:02.323306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4491aab55964f309a2de95a603494b8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:02.324455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4491aab55964f309a2de95a603494b8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:02.340649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4491aab55964f309a2de95a603494b8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:02.401796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c297f6b6-a2e3-4814-b047-6d44a004342e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4491aab55964f309a2de95a603494b8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:02.425536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 834/1656] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:01 2022] DELETE /identity/v3/users/d4491aab55964f309a2de95a603494b8 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:03.243699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:03.245734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:03.283888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50dc677-197e-4c5f-984a-d7d42393aa5b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:03.299584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:03.464851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:03.466095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:03.467390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:03.467685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] PATH_INFO: `/v3/users/78ae9d0f1ba04d10ac840829b4236370` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:03.470189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ae9d0f1ba04d10ac840829b4236370)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:03.475943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:03.635846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ae9d0f1ba04d10ac840829b4236370'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:03.651831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ae9d0f1ba04d10ac840829b4236370'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:03.712183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ae9d0f1ba04d10ac840829b4236370'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:03.712976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ae9d0f1ba04d10ac840829b4236370'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:03.739917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86425bc9-0595-413b-882d-2565ee7429f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ae9d0f1ba04d10ac840829b4236370'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:03.751412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:03.752865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:03.753783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:03.756401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 823/1657] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:03 2022] DELETE /identity/v3/users/78ae9d0f1ba04d10ac840829b4236370 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:03.759738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:03.760092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:03.760474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:03.812210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:03.814801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:03.825807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] MFA Rules not processed for user `d0cef7a208fb403db0c8f590b8e8d7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:03.826477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:03.979849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 824/1658] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:04.002228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:04.005060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:04.116066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:04.125108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 835/1659] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:03 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:04.193508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0974da-73c6-4409-9bd6-a8a30dffa33d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:04.208368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:04.209032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:04.209312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:04.209973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:04.242922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:04.244423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e38472-8bb1-413f-91b1-a4e2f8eb64c0 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:04.276696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 825/1660] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:04.451381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:04.453108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:04.487960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f7e8c-fcd4-4d47-b7aa-864a54a0b5f1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:04.512115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:04.513178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:04.513178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:04.513374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] PATH_INFO: `/v3/projects/b9bffc6bed2b453a969094d233f1aa44` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:04.515570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9bffc6bed2b453a969094d233f1aa44)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:04.524029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: WARNING py.warnings [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 12:29:04.524029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: warnings.warn(msg) Sep 22 12:29:04.524029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: Sep 22 12:29:04.527450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:04.665884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:04.732320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9bffc6bed2b453a969094d233f1aa44', 'request_id': 'req-504fac76-412f-4cc2-9c49-f22b38d4d275'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:04.753027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9bffc6bed2b453a969094d233f1aa44', 'request_id': 'req-504fac76-412f-4cc2-9c49-f22b38d4d275'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:04.761801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9bffc6bed2b453a969094d233f1aa44', 'request_id': 'req-504fac76-412f-4cc2-9c49-f22b38d4d275'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:04.817946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] The token cache is being invalidate because project b9bffc6bed2b453a969094d233f1aa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:04.818249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:04.822006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-504fac76-412f-4cc2-9c49-f22b38d4d275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:04.826975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 836/1661] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:04 2022] DELETE /identity/v3/projects/b9bffc6bed2b453a969094d233f1aa44 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:04.832001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:04.832904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:04.833266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:04.902293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:04.903530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:05.051693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 826/1662] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:05.060744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.061429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.061928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.115887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:05.121949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:05.154997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 837/1663] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:05.156650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.175187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0546ceca-60b0-4905-8717-026a51ef12f0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.189050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.192499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.192972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.193625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.195848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:05.196539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.199520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.268805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 827/1664] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:05.272961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.288591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440109b2-88c3-40fd-9792-6bbb3a8307aa None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.305322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.305710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.306060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.306989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.310467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.321379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.430672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 838/1665] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:05 2022] POST /identity/v3/projects => generated 371 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:05.432137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.452753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac198-5d7d-474d-9963-802651a533ba admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.599353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.600008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.600484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.600960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.602016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.606518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.667106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 828/1666] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:05 2022] POST /identity/v3/users => generated 286 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:05.669124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.692399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54cb4-532a-4907-bc76-9b05751fcc75 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.700088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.700790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.701072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.701344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.702006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.703623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.727758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 839/1667] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:05.732206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.759273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b625b-8bd8-4677-b759-0cd7cf7abac2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.778346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df29f081-165d-4315-a432-699d901889da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.780409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f081-165d-4315-a432-699d901889da admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.781066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f081-165d-4315-a432-699d901889da admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.781656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df29f081-165d-4315-a432-699d901889da admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.786211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29f081-165d-4315-a432-699d901889da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.790116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df29f081-165d-4315-a432-699d901889da admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.821818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 829/1668] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:05.825276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:05.841384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e7deb-f7ee-4ed1-bfef-fbe2930d8fd7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:05.857354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:05.858049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.858335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.858603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] PATH_INFO: `/v3/projects/b33798f3b89546c4be5654eedf8e7376/users/720aae49b118453d9d3641ad30852e57/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:05.895467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33798f3b89546c4be5654eedf8e7376, user_id=720aae49b118453d9d3641ad30852e57, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:05.898290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:05.921637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 840/1669] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:05 2022] PUT /identity/v3/projects/b33798f3b89546c4be5654eedf8e7376/users/720aae49b118453d9d3641ad30852e57/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:05.932904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:05.933184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:05.933448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.010468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] MFA Rules not processed for user `720aae49b118453d9d3641ad30852e57`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:06.011896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:06.079002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:06.134919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 830/1670] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:06.138886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.305317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79abb661-62bd-452e-90e0-cd390f33d151 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.317030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:06.317930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.318424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.318888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.319739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:06.322771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:06.380162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 841/1671] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:06 2022] POST /identity/v3/projects => generated 371 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:06.382414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.397834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc578f-62ab-4b70-b8f6-f084953ad89f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.530731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:06.531789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.533007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.533428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.534417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:06.539715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:06.598584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 831/1672] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:06 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:06.603255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.619525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.626730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:06.632794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75705c9f-0332-48cd-90a1-14c793701fab admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.646550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:06.647158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.647436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.647707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.648376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:06.658840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:06.666932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c71443-8af5-4b29-aed4-91eecf61c7c5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.684542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:06.686364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.687097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.687381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] PATH_INFO: `/v3/projects/e47fbc746f4c4c3c8a1adfd454035fce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.692696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47fbc746f4c4c3c8a1adfd454035fce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:06.699705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:06.700906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 842/1673] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:06.701883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.747511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dfe5f-1403-4a6e-884a-dc7c4380102d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.765065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:06.765687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.766170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.766463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] PATH_INFO: `/v3/projects/9c8b9b887e324cfe9190540d70792136/users/0712c8ce11cc4642a6c015be34e1290d/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.806042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8b9b887e324cfe9190540d70792136, user_id=0712c8ce11cc4642a6c015be34e1290d, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:06.813124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:06.841250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 843/1674] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:06 2022] PUT /identity/v3/projects/9c8b9b887e324cfe9190540d70792136/users/0712c8ce11cc4642a6c015be34e1290d/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:06.846746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:06.852741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47fbc746f4c4c3c8a1adfd454035fce', 'request_id': 'req-c951c2e0-346c-4b2d-95a6-b2e99dc75623'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:06.855537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:06.886530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47fbc746f4c4c3c8a1adfd454035fce', 'request_id': 'req-c951c2e0-346c-4b2d-95a6-b2e99dc75623'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:06.896556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47fbc746f4c4c3c8a1adfd454035fce', 'request_id': 'req-c951c2e0-346c-4b2d-95a6-b2e99dc75623'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:06.920681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] The token cache is being invalidate because project e47fbc746f4c4c3c8a1adfd454035fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:06.921392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:06.922484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c951c2e0-346c-4b2d-95a6-b2e99dc75623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:06.931662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 832/1675] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:06 2022] DELETE /identity/v3/projects/e47fbc746f4c4c3c8a1adfd454035fce => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:06.934292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44493-ef0c-47ad-bbc0-555f6aba40db None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.934292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44493-ef0c-47ad-bbc0-555f6aba40db None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.934292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f44493-ef0c-47ad-bbc0-555f6aba40db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:06.975342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c2f44493-ef0c-47ad-bbc0-555f6aba40db None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:06.976995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c2f44493-ef0c-47ad-bbc0-555f6aba40db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:06.987869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 833/1676] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:06.995899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a32463-2a10-42e3-a3de-1a94721d1a64 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:06.996885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:06.997238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:06.997592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.009721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.009721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.010083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.010841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.012701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.015750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.044974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 844/1677] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:07.046127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:07.048336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.049319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:07.052060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:07.068175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 834/1678] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:07.072673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.080681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c4b2d-02dd-4ffa-a1f3-887d985a19af admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.089580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6c52ba-14b5-4b4d-850e-0f0dacdc9250 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.091006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.091446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.091728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.092191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] PATH_INFO: `/v3/projects/9c8b9b887e324cfe9190540d70792136/users/0712c8ce11cc4642a6c015be34e1290d/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.095466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c8b9b887e324cfe9190540d70792136, user_id=0712c8ce11cc4642a6c015be34e1290d, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.098724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2886c0-44b3-41ca-bbb0-6231e796d0c9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.110257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.110865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.111147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.111403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.112084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:07.116816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.122245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.138153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 845/1679] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:07 2022] PUT /identity/v3/projects/9c8b9b887e324cfe9190540d70792136/users/0712c8ce11cc4642a6c015be34e1290d/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:07.148180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.148528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.148870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.157308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 835/1680] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:07.160976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.183644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d22d986-9297-445e-b94a-e42193a7e1ba admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.211760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:07.228007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] MFA Rules not processed for user `0712c8ce11cc4642a6c015be34e1290d`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:07.229573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:07.326381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.328646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.330307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.330984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.333443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.337401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.389751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 836/1681] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:07 2022] POST /identity/v3/projects => generated 369 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:07.391115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.412008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 846/1682] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:07.424209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.427314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:07.567495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f5fac-d8c5-4104-9188-0f53fd2fd493 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.577819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.578353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8717a9e-f6e6-4d81-b019-e30ac5d79e62 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.578460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.578850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.579363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.580046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.582179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.633401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 837/1683] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:07 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:07.640975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.642732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:07.663284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b4f28-bfc4-4c30-8a85-8160f938acc8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.683592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.684224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.684531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.684832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.685452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.687029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.687159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.687647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.688577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.688860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.711811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 838/1684] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:07.713069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.736228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fb2c97-5a90-4143-8665-7ad20c6cbdda admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.742779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.743406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.743801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.744434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.745088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.748983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d5341-d382-4315-9c90-097479059eea admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.787135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 839/1685] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:07.787135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928f5d-28c2-48c4-8859-7ae218db05ce None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.787466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928f5d-28c2-48c4-8859-7ae218db05ce None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.792258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928f5d-28c2-48c4-8859-7ae218db05ce None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.794544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 840/1686] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:29:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:07.795343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd928f5d-28c2-48c4-8859-7ae218db05ce None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:07.850808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd928f5d-28c2-48c4-8859-7ae218db05ce None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:07.873690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:07.874484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.874569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.875000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.875000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] PATH_INFO: `/v3/projects/c08732a40a8c45c4955f75f03290692a/users/bbb0c02d0ae440a4bbb56f88fb92d47c/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.875922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6fc146-90a7-49c8-927e-c24ebbe64ea1 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.898870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 847/1687] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:07.906584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f248-c394-4883-9e77-1c69645f114d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.906872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f248-c394-4883-9e77-1c69645f114d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.908380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086f248-c394-4883-9e77-1c69645f114d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.922647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08732a40a8c45c4955f75f03290692a, user_id=bbb0c02d0ae440a4bbb56f88fb92d47c, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:07.929015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03c957-31c7-454c-94c5-2d27e69ee4ed admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:07.976231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 841/1688] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:07 2022] PUT /identity/v3/projects/c08732a40a8c45c4955f75f03290692a/users/bbb0c02d0ae440a4bbb56f88fb92d47c/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:07.978248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:07.978723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:07.979133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:07.980934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8086f248-c394-4883-9e77-1c69645f114d None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:07.985936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8086f248-c394-4883-9e77-1c69645f114d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:08.062042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] MFA Rules not processed for user `bbb0c02d0ae440a4bbb56f88fb92d47c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:08.063295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:08.153761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:08.208599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 848/1689] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:08.248706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 842/1690] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:08.253770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f248-c394-4883-9e77-1c69645f114d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:08.392865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:08.400757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:08.442875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211c58c-4c8a-4ca4-8c20-90a76211f2a4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:08.455093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086f248-c394-4883-9e77-1c69645f114d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:08.464533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:08.465173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:08.465455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:08.465769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:08.481041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:08.481681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:08.481963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:08.482240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:08.482881 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:08.484839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:08.516211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:08.518407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:08.526901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:08.545084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 849/1691] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:08 2022] POST /identity/v3/projects => generated 367 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:08.545963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:08.546088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 843/1692] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:08.734032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f921da-c7a6-4c29-9650-9fb94c76611a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:08.748616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:08.749556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:08.749836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:08.750145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:08.756571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:08.758884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d948d-fcf7-47e5-82d2-e3baeec349f3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:08.813581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 850/1693] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:08 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:08.814320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:08.833336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485ebc8-1f90-485c-8bee-7178d8649ae7 service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:08.846271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:08.848233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:08.848659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:08.850942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:08.852413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:08.857218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:08.900482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:08.900988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:08.900988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:08.935313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 844/1694] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:08.940321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:08.987912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] MFA Rules not processed for user `e125a85011fe402ab2d5782d8fd39dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:08.996519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3acd5-2c22-496c-9834-891ca666173a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:08.997019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.006805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:09.007538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.007822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.008097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] PATH_INFO: `/v3/projects/bd1d7e09d993402187eecc1f0ade73cc/users/0c652b5698e24d43afef5e6c7861151f/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.065093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1d7e09d993402187eecc1f0ade73cc, user_id=0c652b5698e24d43afef5e6c7861151f, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:09.078063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:09.103633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 845/1695] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:08 2022] PUT /identity/v3/projects/bd1d7e09d993402187eecc1f0ade73cc/users/0c652b5698e24d43afef5e6c7861151f/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:09.107196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:09.133748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 851/1696] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:09.135139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:09.148882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719af13-a8aa-4b1a-8b78-04ceabdbb3a0 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:09.229481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73db8602-919e-4826-988d-07767dae8a0b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:09.250897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:09.251953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.253103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.253388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] PATH_INFO: `/v3/users/720aae49b118453d9d3641ad30852e57` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.255787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=720aae49b118453d9d3641ad30852e57)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:09.257755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:09.264081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:09.264729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.265136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.265423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.266069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:09.267670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf67af-9ddf-4660-80b2-af07a570f85e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:09.297183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 852/1697] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:09 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:09.306779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30216-489b-41c5-9116-c588c5ff6a3f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.307120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30216-489b-41c5-9116-c588c5ff6a3f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.307354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb30216-489b-41c5-9116-c588c5ff6a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.351919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-adb30216-489b-41c5-9116-c588c5ff6a3f None None] MFA Rules not processed for user `e125a85011fe402ab2d5782d8fd39dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:09.353259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-adb30216-489b-41c5-9116-c588c5ff6a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.371208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:09.371999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720aae49b118453d9d3641ad30852e57'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:09.384510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720aae49b118453d9d3641ad30852e57'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:09.408641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720aae49b118453d9d3641ad30852e57'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:09.411745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720aae49b118453d9d3641ad30852e57'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:09.428879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720aae49b118453d9d3641ad30852e57'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:09.443899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 846/1698] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:09 2022] DELETE /identity/v3/users/720aae49b118453d9d3641ad30852e57 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:09.445179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:09.450450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.465714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:09.485223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 853/1699] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:09.486727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940e0fd-c1b7-4780-bd3d-c50401ae8d41 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.488559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940e0fd-c1b7-4780-bd3d-c50401ae8d41 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.496069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940e0fd-c1b7-4780-bd3d-c50401ae8d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.534299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6940e0fd-c1b7-4780-bd3d-c50401ae8d41 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:09.538855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6940e0fd-c1b7-4780-bd3d-c50401ae8d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.597194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bda3f5-7458-4b4c-9b21-d0f055bdc7cd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:09.610820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:09.611509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.611785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.612060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] PATH_INFO: `/v3/projects/bd1d7e09d993402187eecc1f0ade73cc/users/0c652b5698e24d43afef5e6c7861151f/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.616951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1d7e09d993402187eecc1f0ade73cc, user_id=0c652b5698e24d43afef5e6c7861151f, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:09.623614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:09.659828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 847/1700] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:09 2022] PUT /identity/v3/projects/bd1d7e09d993402187eecc1f0ade73cc/users/0c652b5698e24d43afef5e6c7861151f/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:09.660995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:09.666286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.682184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 854/1701] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:09.682184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.683121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.683404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.750867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] MFA Rules not processed for user `0c652b5698e24d43afef5e6c7861151f`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:09.752637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.836031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c7bb5-3946-4f8b-8efa-f3a4bc78073c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:09.853658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:09.854673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:09.855386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:09.855677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:09.862088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:09.920476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 855/1702] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:09.928403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:09.933578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.002085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.004547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.031720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 848/1703] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:10.032720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:10.035384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:10.036708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.072213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcbf7b-1400-4892-93df-82467c8fcdf6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:10.094599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:10.095236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.095514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.095786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.120640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:10.183309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f83e2-d650-488a-81cb-461d39e22349 service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:10.202249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:10.203720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.206291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.206291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.248850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.250329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339c98a-19c0-4d04-a103-ed9e7c17a7db service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.250694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.258231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f949e-36e6-4e2d-94ec-ffc3af17f724 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.265547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 856/1704] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:10.266802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.267087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.267353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.268771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 857/1705] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:10.269346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:10.281310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 849/1706] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:10.282839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.283194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.283537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.297221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.306919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:10.308515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.359126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743bb7d-4008-4ba5-a59a-73d6086d2a81 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:10.378533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:10.379510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.379961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.380430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] PATH_INFO: `/v3/users/eea4148a487046a384ffab5d7d698b54` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.383249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea4148a487046a384ffab5d7d698b54)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.385436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.409605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 850/1707] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:10.412497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:10.437518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af5756b-e047-4095-b125-5f6d98732500 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:10.447498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:10.450107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.450932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.452104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.455995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.471586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea4148a487046a384ffab5d7d698b54'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:10.490060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea4148a487046a384ffab5d7d698b54'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:10.490980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea4148a487046a384ffab5d7d698b54'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:10.503079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea4148a487046a384ffab5d7d698b54'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:10.523833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea4148a487046a384ffab5d7d698b54'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:10.535719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 858/1708] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:10 2022] DELETE /identity/v3/users/eea4148a487046a384ffab5d7d698b54 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:10.536734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:10.538433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.605226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.606685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe09ad-a510-49c8-905b-6eed177d6da3 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.629296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 851/1709] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:10.688372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d005e-0989-4f63-b51d-e85210d90862 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:10.704377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:10.705207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.705625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.706506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.710814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:10.741469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:10.744229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:10.746833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcbf94-e1fc-48ed-b407-d95985187967 service placement] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:10.770350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 859/1710] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:10.995020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:10.995494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:10.995937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:10.997470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 852/1711] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 22 12:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:11.007252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:11.007781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:11.008184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:11.124585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:11.126770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:11.129640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:11.132670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:11.290190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 860/1712] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:29:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:11.296719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2971acb-19e0-4a7c-897c-deeaecf899e2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:11.303434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:11.304927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:11.313078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:11.314095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:11.314593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:11.315059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] PATH_INFO: `/v3/projects/c3d5c01356eb45f9b87fddc5526b6de1` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:11.317311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3d5c01356eb45f9b87fddc5526b6de1)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:11.327054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64a08-9977-4c47-8ecf-fbe77aa08f14 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:11.327795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:11.335894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:11.336622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:11.336813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:11.337082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:11.371018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:11.372983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:11.408204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 861/1713] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:11.427164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:11.541072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3d5c01356eb45f9b87fddc5526b6de1', 'request_id': 'req-265147f6-ffb0-4b13-97a0-0f7f63883ce6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.561392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3d5c01356eb45f9b87fddc5526b6de1', 'request_id': 'req-265147f6-ffb0-4b13-97a0-0f7f63883ce6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.575910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3d5c01356eb45f9b87fddc5526b6de1', 'request_id': 'req-265147f6-ffb0-4b13-97a0-0f7f63883ce6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.601405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] The token cache is being invalidate because project c3d5c01356eb45f9b87fddc5526b6de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:11.602515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.603684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.605897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 853/1714] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:11 2022] DELETE /identity/v3/projects/c3d5c01356eb45f9b87fddc5526b6de1 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:11.609381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:11.630566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170ef98-762d-49a0-9ad4-5b5ad806e1f6 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:11.772088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:11.772804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:11.773082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:11.773354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] PATH_INFO: `/v3/users/d0cef7a208fb403db0c8f590b8e8d7a0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:11.775160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0cef7a208fb403db0c8f590b8e8d7a0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:11.777676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:11.917516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0cef7a208fb403db0c8f590b8e8d7a0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.933576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0cef7a208fb403db0c8f590b8e8d7a0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.934241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0cef7a208fb403db0c8f590b8e8d7a0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.949750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0cef7a208fb403db0c8f590b8e8d7a0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.993305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0cef7a208fb403db0c8f590b8e8d7a0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:11.997358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:12.014417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 862/1715] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:11 2022] DELETE /identity/v3/users/d0cef7a208fb403db0c8f590b8e8d7a0 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:12.303197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:12.304678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:12.475101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265147f6-ffb0-4b13-97a0-0f7f63883ce6 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:12.674177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:12.677830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:12.704525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:12.704525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:12.704525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:12.704525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:12.717768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:12.747870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:12.752334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b606540-b59a-4ba9-9a41-950e8dfe43c9 service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:12.778877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 854/1716] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:12.882425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40e8f9-df0c-4791-b49a-ab2fcd642745 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:12.890203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:12.891224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:12.891702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:12.892237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] PATH_INFO: `/v3/projects/cd403ec87b894c1cbf6adf020919ac66` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:12.895704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd403ec87b894c1cbf6adf020919ac66)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:12.898667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:13.130404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd403ec87b894c1cbf6adf020919ac66', 'request_id': 'req-a96ed267-1462-40cf-be23-de36bad658ab'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:13.143766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd403ec87b894c1cbf6adf020919ac66', 'request_id': 'req-a96ed267-1462-40cf-be23-de36bad658ab'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:13.151888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd403ec87b894c1cbf6adf020919ac66', 'request_id': 'req-a96ed267-1462-40cf-be23-de36bad658ab'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:13.183132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] The token cache is being invalidate because project cd403ec87b894c1cbf6adf020919ac66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:13.183605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:13.189501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a96ed267-1462-40cf-be23-de36bad658ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:13.190446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 863/1717] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:12 2022] DELETE /identity/v3/projects/cd403ec87b894c1cbf6adf020919ac66 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:13.195569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:13.195800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:13.197941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:13.279121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:13.284454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:13.434173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:13.623842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 855/1718] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:13.626131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:13.626779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:13.627281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:13.690263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:13.695413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:13.723949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 864/1719] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:13.726388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:13.745632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb4b8b-d6c0-4748-b2b7-a4ca6fdc1d97 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:13.766423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:13.767614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:13.772843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:13.773738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:13.777106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:13.779090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:13.786096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:13.853132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 856/1720] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:13.856848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:13.892913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6adcbc-a1f3-4f0d-bc73-446dd763ba76 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:13.937149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:13.937149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:13.937149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:13.947525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:13.947525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:13.955092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:14.084580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 865/1721] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:13 2022] POST /identity/v3/projects => generated 371 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:14.089098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.107779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3857cde-4343-49f1-ac5a-da737a3bcf8b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:14.311578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:14.312203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.312566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.312850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.313492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:14.318494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:14.352033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 857/1722] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:14 2022] POST /identity/v3/users => generated 286 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:14.356464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.377097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b85fc39-f431-4a2c-a9f7-4f6f0e7bbac4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:14.397883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:14.398525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.398802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.401601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.402294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:14.403920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:14.430264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 866/1723] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:14.434297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.448946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf9508-65d7-4ef6-a0aa-3269a7990da7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:14.461611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:14.462622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.463143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.463613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.464514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:14.467942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96234d27-b2e7-4549-8ef6-bf19544bc925 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:14.497045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 858/1724] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:14 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:14.500381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.518400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a0194-8339-404f-9749-b9dd0d199a5d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:14.537526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:14.538213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.538491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.538760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] PATH_INFO: `/v3/projects/cd67711e9def4c7f89a21fc1d464a055/users/44ab3ba5be3b45948167d69dcc3161eb/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.601124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd67711e9def4c7f89a21fc1d464a055, user_id=44ab3ba5be3b45948167d69dcc3161eb, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:14.604092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:14.637042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 867/1725] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:14 2022] PUT /identity/v3/projects/cd67711e9def4c7f89a21fc1d464a055/users/44ab3ba5be3b45948167d69dcc3161eb/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:14.647219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.648250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.648722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.699784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.701621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:14.703991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] MFA Rules not processed for user `44ab3ba5be3b45948167d69dcc3161eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:14.705237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:14.776649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:14.853005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 859/1726] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:14.858134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:14.923485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299699c3-c616-4722-94fe-05eefe0b6ae5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:14.926464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:14.969077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:14.969745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:14.970024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:14.970301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] PATH_INFO: `/v3/projects/b33798f3b89546c4be5654eedf8e7376` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:14.972200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33798f3b89546c4be5654eedf8e7376)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:14.974706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:15.020826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99369-332f-4e03-9a42-07b4e984e421 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:15.031367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:15.033555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:15.033930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:15.034243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:15.050419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:15.059623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:15.136340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 860/1727] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:14 2022] POST /identity/v3/projects => generated 369 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:15.138548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:15.180958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33798f3b89546c4be5654eedf8e7376', 'request_id': 'req-fb2d19c1-8a55-4309-8296-5b319eab0b5d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.205557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33798f3b89546c4be5654eedf8e7376', 'request_id': 'req-fb2d19c1-8a55-4309-8296-5b319eab0b5d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.223408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33798f3b89546c4be5654eedf8e7376', 'request_id': 'req-fb2d19c1-8a55-4309-8296-5b319eab0b5d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.257753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] The token cache is being invalidate because project b33798f3b89546c4be5654eedf8e7376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:15.262048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.264130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.267683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 868/1728] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:14 2022] DELETE /identity/v3/projects/b33798f3b89546c4be5654eedf8e7376 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:15.274117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:15.284446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:15.295264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ded45f-4123-46ff-9e6c-3dc415c7743f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:15.518610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:15.519144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:15.522131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:15.522437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:15.523092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:15.530050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:15.543170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d19c1-8a55-4309-8296-5b319eab0b5d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:15.564186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:15.564878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:15.565156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:15.565437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] PATH_INFO: `/v3/users/0712c8ce11cc4642a6c015be34e1290d` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:15.567569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0712c8ce11cc4642a6c015be34e1290d)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:15.569687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:15.605533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 861/1729] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:15 2022] POST /identity/v3/users => generated 291 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:15.607154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:15.615152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:15.673305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8fa9e-ede9-4aed-a33d-1719afb9b172 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:15.674346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0712c8ce11cc4642a6c015be34e1290d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.693571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:15.694821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:15.696705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:15.696992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:15.698295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0712c8ce11cc4642a6c015be34e1290d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.699029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0712c8ce11cc4642a6c015be34e1290d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.700884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:15.704505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:15.722970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0712c8ce11cc4642a6c015be34e1290d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.730350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:15.757952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0712c8ce11cc4642a6c015be34e1290d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:15.770218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 869/1730] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:15 2022] DELETE /identity/v3/users/0712c8ce11cc4642a6c015be34e1290d => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:15.793438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 862/1731] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:15.794779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:15.796436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.019502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c861f1-bab7-498b-91fe-7c27d6c09fdd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:16.034949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:16.035708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.038951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.039161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] PATH_INFO: `/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf/users/41ba50b350f949d98fd9615d38b1df83/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.079269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4da2fa32d64563b26e320e62a1ddbf, user_id=41ba50b350f949d98fd9615d38b1df83, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:16.087960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c359eb8-609a-4eb5-8739-5e3e206308cd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:16.129638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 870/1732] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:15 2022] PUT /identity/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf/users/41ba50b350f949d98fd9615d38b1df83/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:16.142994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:16.146696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699cdde-4ebf-452c-ace6-37f908d0d149 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.147347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699cdde-4ebf-452c-ace6-37f908d0d149 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.147347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699cdde-4ebf-452c-ace6-37f908d0d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.163525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f7f75-ec17-46d5-b77a-b571ea175ec3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:16.226208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e699cdde-4ebf-452c-ace6-37f908d0d149 None None] MFA Rules not processed for user `0847f00afbde44068fcf1cedbce24be5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:16.230139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e699cdde-4ebf-452c-ace6-37f908d0d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.273028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:16.317964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:16.321391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.321617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.322297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.323677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:16.329281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:16.367343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 863/1733] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:16.371922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:16.414293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182c46f-94bc-49c9-91cd-72ec838c57c3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:16.426684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:16.427811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.428104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.428409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] PATH_INFO: `/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf/users/41ba50b350f949d98fd9615d38b1df83/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.433784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f4da2fa32d64563b26e320e62a1ddbf, user_id=41ba50b350f949d98fd9615d38b1df83, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:16.438466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b635db-1175-42f6-bc17-e4e2ddacd60b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:16.458068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 864/1734] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:16 2022] PUT /identity/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf/users/41ba50b350f949d98fd9615d38b1df83/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:16.469395 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.469768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.472145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.503902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 871/1735] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:16.505817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:16.506324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:16.506504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:16.561717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] MFA Rules not processed for user `0847f00afbde44068fcf1cedbce24be5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:16.564307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.569363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] MFA Rules not processed for user `41ba50b350f949d98fd9615d38b1df83`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:16.571469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.736813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:16.744937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 865/1736] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:16.747108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:16.749222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.772982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 872/1737] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:16.774620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:16.794925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:16.981284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def5b307-57b8-49e5-98c8-0a3406c125a3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:17.000221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc44f1c-68e6-46aa-83c9-84cc08940ffb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:17.001641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:17.002213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:17.002508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:17.004335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] PATH_INFO: `/v3/users/bbb0c02d0ae440a4bbb56f88fb92d47c` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:17.005893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb0c02d0ae440a4bbb56f88fb92d47c)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:17.007851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:17.012794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:17.013433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:17.013708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:17.013981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:17.042103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:17.043614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:17.090261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 873/1738] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:17.091116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:17.098795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:17.155549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb0c02d0ae440a4bbb56f88fb92d47c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:17.168316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:17.178476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb0c02d0ae440a4bbb56f88fb92d47c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:17.222299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb0c02d0ae440a4bbb56f88fb92d47c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:17.223195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb0c02d0ae440a4bbb56f88fb92d47c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:17.282032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-03d67923-6488-4e16-a54d-7c935e1c5b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb0c02d0ae440a4bbb56f88fb92d47c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:17.308468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 866/1739] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:16 2022] DELETE /identity/v3/users/bbb0c02d0ae440a4bbb56f88fb92d47c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:17.308468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d47-d38e-4a1b-881f-6df8514ff51f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:17.308468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d47-d38e-4a1b-881f-6df8514ff51f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:17.308468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a881d47-d38e-4a1b-881f-6df8514ff51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:17.419083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ba08c-fc9b-4d08-997f-82b4d9e369cc service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:17.424587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1a881d47-d38e-4a1b-881f-6df8514ff51f None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:17.426936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1a881d47-d38e-4a1b-881f-6df8514ff51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:17.686578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:17.688990 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 867/1740] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:17.690156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:17.690387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:17.690661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:17.692599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:17.692876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:17.693146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:17.700467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:17.778125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:17.783031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:17.830401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:17.945339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:17.952650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:17.983089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 874/1741] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 892 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:17.984001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:17.989759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:18.026715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 868/1742] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:18.037699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:18.039403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:18.162265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae483aad-133a-4d70-92f0-1afb3abe7ebd service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:18.171945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:18.173044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:18.174041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:18.174620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:18.181948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:18.183307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:18.330015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc0a8e6-c43b-4b9a-b623-e6dad28bfa83 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:18.336387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:18.337189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:18.337769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:18.338047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:18.338352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:18.340403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:18.355215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:18.359807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eae43f0-91bf-4a10-9ddf-3741c812c637 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:18.370093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 875/1743] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:18.372236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:18.378274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:18.397675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 869/1744] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:29:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:29:18.493690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37089f2-b8e3-40d9-9850-c07ed6fd1270 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:18.498550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:18.499674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:18.500147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:18.501475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:18.527335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:18.549108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:18.588435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 876/1745] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:18.827576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda8f9-4dee-48ba-9dc5-d51066ae6fa5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:18.828132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda8f9-4dee-48ba-9dc5-d51066ae6fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:18.828227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fda8f9-4dee-48ba-9dc5-d51066ae6fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:18.950346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-66fda8f9-4dee-48ba-9dc5-d51066ae6fa5 None None] MFA Rules not processed for user `55a7f7ba0b27479f8d1e68d808637d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:18.951739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-66fda8f9-4dee-48ba-9dc5-d51066ae6fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:19.163190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 870/1746] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:19.276687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:19.279337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:19.305523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a8ea8-836d-4fd9-aea8-088a4ae38fbb service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:19.334440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:19.335437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:19.335927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:19.336470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:19.358155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:19.360182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf1826-ce98-41ba-ac6f-94dd2f1f71f5 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:19.381697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 877/1747] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:19.462995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:19.463582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:19.464001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:19.466048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 871/1748] 10.222.0.33 () {54 vars in 912 bytes} [Thu Sep 22 12:29:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:19.474990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4780e-5213-472c-bab0-0837431463ad None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:19.475887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4780e-5213-472c-bab0-0837431463ad None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:19.477330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4780e-5213-472c-bab0-0837431463ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:19.523042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-dae4780e-5213-472c-bab0-0837431463ad None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:19.524471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dae4780e-5213-472c-bab0-0837431463ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:19.539999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 878/1749] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:29:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:20.156877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:20.158270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:20.202651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56fcc5-9d7e-4474-a873-b8ff71c771b2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:20.215198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4780e-5213-472c-bab0-0837431463ad None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:20.216408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dae4780e-5213-472c-bab0-0837431463ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:20.227756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:20.228061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:20.228382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:20.228658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] PATH_INFO: `/v3/users/368a8b674a824ace996f84d0454444da` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:20.241192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368a8b674a824ace996f84d0454444da)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:20.258964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:20.277130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4780e-5213-472c-bab0-0837431463ad None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:20.312946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:20.313665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:20.313951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:20.323978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] PATH_INFO: `/v3/projects/9c8b9b887e324cfe9190540d70792136` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:20.329128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c8b9b887e324cfe9190540d70792136)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:20.339696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:20.351126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:20.469865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c8b9b887e324cfe9190540d70792136', 'request_id': 'req-9db2eae0-67ac-4458-9b85-229b698fc119'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.493297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c8b9b887e324cfe9190540d70792136', 'request_id': 'req-9db2eae0-67ac-4458-9b85-229b698fc119'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.504659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c8b9b887e324cfe9190540d70792136', 'request_id': 'req-9db2eae0-67ac-4458-9b85-229b698fc119'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.528183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368a8b674a824ace996f84d0454444da'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.536101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] The token cache is being invalidate because project 9c8b9b887e324cfe9190540d70792136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:20.539596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.540796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.547995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 879/1750] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:20 2022] DELETE /identity/v3/projects/9c8b9b887e324cfe9190540d70792136 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:20.548944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:20.556750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:20.569082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368a8b674a824ace996f84d0454444da'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.640351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368a8b674a824ace996f84d0454444da'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.645676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368a8b674a824ace996f84d0454444da'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.679761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-9788724f-4bca-46b4-ad84-aee8baed9226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368a8b674a824ace996f84d0454444da'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:20.716641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 872/1751] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:20 2022] DELETE /identity/v3/users/368a8b674a824ace996f84d0454444da => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:20.721191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2eae0-67ac-4458-9b85-229b698fc119 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:20.724869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb199ad-5ec4-421f-8b1c-5f13cb0a3677 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:20.729425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb199ad-5ec4-421f-8b1c-5f13cb0a3677 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:20.729794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb199ad-5ec4-421f-8b1c-5f13cb0a3677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:20.824091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-efb199ad-5ec4-421f-8b1c-5f13cb0a3677 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:20.825518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-efb199ad-5ec4-421f-8b1c-5f13cb0a3677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:20.860925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:20.997144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:20.997144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:20.997456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:20.997793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.012266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:21.049646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.051979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6238d4a-5450-4bb4-a09d-1ee8a5ed06c9 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.059197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 873/1752] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:21.060830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.061187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.061551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.074606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 880/1753] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:21.076099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.079121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.079712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.125947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:21.127065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:21.165413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 874/1754] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:21.169033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:21.185274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:21.197747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.199616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:21.210924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 881/1755] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:21.215368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.220387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c3bd0-d0dd-4e29-a946-ebcc1c83e969 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.234325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.235352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.237471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.239206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.242735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda88eee-508e-4b4e-9833-48a451e0d383 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.270965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.271768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.272224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.273001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.276829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:21.277889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.279265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.284968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.287964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.320106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 875/1756] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:21.322603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 882/1757] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:21.327132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.347035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b7b82f-15da-4c0e-9e97-d2fd20737b2c service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.360398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.361139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.362275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.367473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.368289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.371324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.452834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 876/1758] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:21 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:21.457737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.477806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db4951-2ee5-4acb-b310-47b3c41c9347 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.559711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:21.621253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.621902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.622179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.622446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.623091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.625059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.663814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 883/1759] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:21 2022] POST /identity/v3/users => generated 296 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:21.667863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.686696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace9bbc-cefb-49f8-b198-742c0b90555b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.695607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.696594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.697273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.697912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.699294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.702002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.718440 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 877/1760] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:21.723020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.738437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d4c5-7a4b-42ba-b634-33703305274a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.753298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.754135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.754637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.754763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.755349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.757122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8273e64-4f87-4fa6-a261-aad5c87e6309 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.784402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 884/1761] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:21.787729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:21.805097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398f585-0ebd-4ad5-9643-1b2c0bb7acd4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:21.821657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:21.822577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.823051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.826941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] PATH_INFO: `/v3/projects/37bb9db0546f4906806214f1f1131fdf/users/4383d2dfd0224f50be0569ffdfa6e3b7/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:21.885856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37bb9db0546f4906806214f1f1131fdf, user_id=4383d2dfd0224f50be0569ffdfa6e3b7, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:21.897612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:21.945537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 878/1762] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:21 2022] PUT /identity/v3/projects/37bb9db0546f4906806214f1f1131fdf/users/4383d2dfd0224f50be0569ffdfa6e3b7/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:21.956279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d204-648e-4b96-868b-aa9fd25b3c34 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:21.956279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d204-648e-4b96-868b-aa9fd25b3c34 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:21.957143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d730d204-648e-4b96-868b-aa9fd25b3c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.025833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d730d204-648e-4b96-868b-aa9fd25b3c34 None None] MFA Rules not processed for user `4383d2dfd0224f50be0569ffdfa6e3b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:22.027307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d730d204-648e-4b96-868b-aa9fd25b3c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:22.174895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 885/1763] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:22.179378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:22.310779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:22.311314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:22.311314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.361628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc980a6-c1b3-4977-ac7d-f0fec65a8506 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:22.366663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:22.367283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:22.367562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:22.367840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.368496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:22.370439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:22.397923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] MFA Rules not processed for user `49c772b0c2bb49a4ab4048d980bd20a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:22.408322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:22.441986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 879/1764] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:22 2022] POST /identity/v3/projects => generated 391 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:22.443103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:22.526075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:22.628528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 886/1765] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:22.670253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aadd94-f487-4b42-a841-46045e36e92e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:22.677739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:22.679854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:22.681586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:22.682088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.682960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:22.686340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:22.732377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 880/1766] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:22 2022] POST /identity/v3/users => generated 302 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:22.734108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:22.756660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271ac1f-e8ad-4cd4-bf14-a08dcfe93840 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:22.773864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:22.777926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:22.779744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:22.780214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.780927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:22.790641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:22.842675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 887/1767] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:22.844043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:22.866106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c526e-68a3-482b-8b76-1af4fdcd5aa0 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:22.885041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:22.889517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:22.891507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:22.891811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] PATH_INFO: `/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1/users/184d77907d9a402c8526c0f758bf9b63/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:22.974556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155bf88ed03e48d7a2f9323ad7dd24e1, user_id=184d77907d9a402c8526c0f758bf9b63, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:22.974556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:23.065603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 881/1768] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:22 2022] PUT /identity/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1/users/184d77907d9a402c8526c0f758bf9b63/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:23.069741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:23.103166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:23.120786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:23.139250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d313b-e4fa-4d55-84f7-7bf56443c531 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:23.213890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:23.421360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361fffa-a66b-4681-8a0b-172c228bf2f9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:23.429631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:23.432069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:23.433975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:23.434429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:23.436238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:23.440816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:23.442326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:23.442748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:23.443167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] PATH_INFO: `/v3/projects/c08732a40a8c45c4955f75f03290692a` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:23.444481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08732a40a8c45c4955f75f03290692a)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:23.444724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:23.446694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:23.500593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 888/1769] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:23.500593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:23.525364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:23.631892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08732a40a8c45c4955f75f03290692a', 'request_id': 'req-5122b4cd-7204-4ff5-9905-7db623b3c282'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:23.649334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08732a40a8c45c4955f75f03290692a', 'request_id': 'req-5122b4cd-7204-4ff5-9905-7db623b3c282'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:23.656769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08732a40a8c45c4955f75f03290692a', 'request_id': 'req-5122b4cd-7204-4ff5-9905-7db623b3c282'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:23.680760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] The token cache is being invalidate because project c08732a40a8c45c4955f75f03290692a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:23.680987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:23.683155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:23.690506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 882/1770] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:23 2022] DELETE /identity/v3/projects/c08732a40a8c45c4955f75f03290692a => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:23.691562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:23.697339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:23.748562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:23.806717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbd456-f72f-4867-825c-51c17d8162d5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:23.917083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122b4cd-7204-4ff5-9905-7db623b3c282 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:23.955281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:23.956145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:23.957537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:23.957935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] PATH_INFO: `/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1/users/184d77907d9a402c8526c0f758bf9b63/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:23.969106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155bf88ed03e48d7a2f9323ad7dd24e1, user_id=184d77907d9a402c8526c0f758bf9b63, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:23.980326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa76760-3e4b-44e1-b707-595dea735b93 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:23.980909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:23.981692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:23.982137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:23.982547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:23.988429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.069512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 883/1771] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:23 2022] PUT /identity/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1/users/184d77907d9a402c8526c0f758bf9b63/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:24.071966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448b0fe-e79e-4298-a5e3-ee1509e1aa3f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:24.072726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448b0fe-e79e-4298-a5e3-ee1509e1aa3f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:24.073272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448b0fe-e79e-4298-a5e3-ee1509e1aa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:24.104829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:24.133869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a448b0fe-e79e-4298-a5e3-ee1509e1aa3f None None] MFA Rules not processed for user `0847f00afbde44068fcf1cedbce24be5`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:24.141591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a448b0fe-e79e-4298-a5e3-ee1509e1aa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.150053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:24.152616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:24.198466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 889/1772] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:24.202531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:24.210511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.294143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c051c03-02a8-4cfe-86b5-ab502e32cf10 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:24.439287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 884/1773] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:24.439287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cddf8-feb4-4208-bf0c-0108ec359bfe None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:24.439884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cddf8-feb4-4208-bf0c-0108ec359bfe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:24.444337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9cddf8-feb4-4208-bf0c-0108ec359bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:24.518978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7d9cddf8-feb4-4208-bf0c-0108ec359bfe None None] MFA Rules not processed for user `55a7f7ba0b27479f8d1e68d808637d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:24.524178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7d9cddf8-feb4-4208-bf0c-0108ec359bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.554779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:24.556003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:24.568717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:24.572239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] PATH_INFO: `/v3/users/0c652b5698e24d43afef5e6c7861151f` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:24.575381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c652b5698e24d43afef5e6c7861151f)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:24.584139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:24.707012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c652b5698e24d43afef5e6c7861151f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:24.719582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c652b5698e24d43afef5e6c7861151f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:24.720867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c652b5698e24d43afef5e6c7861151f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:24.734575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c652b5698e24d43afef5e6c7861151f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:24.748868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 885/1774] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:24.750198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:24.750473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:24.750764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:24.751992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c652b5698e24d43afef5e6c7861151f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:24.761160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 890/1775] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:24 2022] DELETE /identity/v3/users/0c652b5698e24d43afef5e6c7861151f => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:24.763445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:24.780968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.814171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] MFA Rules not processed for user `184d77907d9a402c8526c0f758bf9b63`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:24.815480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:24.964364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:25.037240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 886/1776] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:24 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:25.047824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:25.050759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:25.070083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72feef3-1a7d-4911-97f0-d566fadcab63 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:25.085880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:25.086128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:25.086227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:25.086544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] PATH_INFO: `/v3/projects/763a056f453c4b9e8c649907c9f0b4ce` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:25.087743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=763a056f453c4b9e8c649907c9f0b4ce)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:25.089935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:25.121448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:25.185476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f4889d-5737-4a35-b8f1-55cb523e9ee0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:25.194829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:25.195618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:25.196033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:25.196348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:25.220886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:25.222621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:25.265453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 887/1777] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:25.276678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763a056f453c4b9e8c649907c9f0b4ce', 'request_id': 'req-dece2a2a-49d0-4111-ade8-89ab180ebc52'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.291676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763a056f453c4b9e8c649907c9f0b4ce', 'request_id': 'req-dece2a2a-49d0-4111-ade8-89ab180ebc52'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.338537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763a056f453c4b9e8c649907c9f0b4ce', 'request_id': 'req-dece2a2a-49d0-4111-ade8-89ab180ebc52'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.391203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:25.393264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:25.406090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] The token cache is being invalidate because project 763a056f453c4b9e8c649907c9f0b4ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:25.407113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.409969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.422146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 891/1778] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:24 2022] DELETE /identity/v3/projects/763a056f453c4b9e8c649907c9f0b4ce => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:25.422276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c81c11-b1a8-4d59-ab2f-836b9920b95f service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:25.423275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:25.425117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:25.568946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece2a2a-49d0-4111-ade8-89ab180ebc52 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:25.575047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:25.575650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:25.576998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:25.578024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:25.578521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:25.579587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:25.581965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:25.582507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:25.582826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] PATH_INFO: `/v3/users/1926018aaeed478f953b541c37a43885` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:25.600018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1926018aaeed478f953b541c37a43885)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:25.602847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:25.711705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1926018aaeed478f953b541c37a43885'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.723924 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1926018aaeed478f953b541c37a43885'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.724938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1926018aaeed478f953b541c37a43885'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.735399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1926018aaeed478f953b541c37a43885'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.758898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1926018aaeed478f953b541c37a43885'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:25.767813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 892/1779] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:25 2022] DELETE /identity/v3/users/1926018aaeed478f953b541c37a43885 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:25.772320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:25.773901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:25.800154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:25.839107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:25.840858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4945e38-9be6-4bfb-afaa-ebf58c6ebda4 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:25.862660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 888/1780] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:25.973409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc2c2c-3b54-4559-abd1-1136f8bff38f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:25.990454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:25.991783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:25.992393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:25.993065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] PATH_INFO: `/v3/projects/bd1d7e09d993402187eecc1f0ade73cc` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:25.997418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1d7e09d993402187eecc1f0ade73cc)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:26.003636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:26.123839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1d7e09d993402187eecc1f0ade73cc', 'request_id': 'req-4907170c-cf38-430b-bb96-3e1ffe3e213c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:26.137662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1d7e09d993402187eecc1f0ade73cc', 'request_id': 'req-4907170c-cf38-430b-bb96-3e1ffe3e213c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:26.145806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1d7e09d993402187eecc1f0ade73cc', 'request_id': 'req-4907170c-cf38-430b-bb96-3e1ffe3e213c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:26.174704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] The token cache is being invalidate because project bd1d7e09d993402187eecc1f0ade73cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:26.175007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:26.176673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4907170c-cf38-430b-bb96-3e1ffe3e213c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:26.179831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 893/1781] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:25 2022] DELETE /identity/v3/projects/bd1d7e09d993402187eecc1f0ade73cc => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:26.185317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:26.185661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:26.186159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:26.231539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:26.232885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:26.345936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:26.403177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:26.403662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:26.404070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:26.407736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 889/1782] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:26.477818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:26.481296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:26.526798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:26.527625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 894/1783] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:26.543427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38420f55-0cc4-4732-8e52-bce92866d582 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:26.565691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:26.567265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:26.569499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:26.569924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:26.570746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:26.572695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:26.574345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:26.602839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 890/1784] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:29:26.607102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:26.622806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53047b8d-e449-4a3a-9c1b-782368ff1d38 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:26.641961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:26.642740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:26.643129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:26.643511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:26.645498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:26.652499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:26.705241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:26.709360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:26.767750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af2017c-cc14-40ed-98d2-cef59d95830b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:26.787698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 895/1785] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:26 2022] POST /identity/v3/projects => generated 379 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:26.790949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.023692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74beb90-925f-478b-a77e-8cab8dd6f910 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:27.031308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:27.032198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.032672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.049189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.051035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:27.054820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492f32e-1d29-4e8b-ae47-bba436504989 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:27.065624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:27.067792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.072030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.078964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] PATH_INFO: `/v3/projects/836bc3b6063446b4ae19121da3539b06` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.088552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=836bc3b6063446b4ae19121da3539b06)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:27.101061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:27.131750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 896/1786] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:26 2022] POST /identity/v3/users => generated 290 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:27.133313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.133592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.133939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.155627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:27.166186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:27.169120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.302495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 897/1787] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 22 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:27.303055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.327459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a5670b-52fe-4f2e-b898-4b69f67f5543 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:27.389420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836bc3b6063446b4ae19121da3539b06', 'request_id': 'req-49fbf211-45a6-4dff-8dc8-39f871868cbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:27.419629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836bc3b6063446b4ae19121da3539b06', 'request_id': 'req-49fbf211-45a6-4dff-8dc8-39f871868cbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:27.445942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836bc3b6063446b4ae19121da3539b06', 'request_id': 'req-49fbf211-45a6-4dff-8dc8-39f871868cbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:27.488796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] The token cache is being invalidate because project 836bc3b6063446b4ae19121da3539b06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:27.489349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:27.491999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:27.498191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 891/1788] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:26 2022] DELETE /identity/v3/projects/836bc3b6063446b4ae19121da3539b06 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:27.499144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.501816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.539458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:27.542793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.545430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.545903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.548023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:27.552400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f206d-a0c2-46a8-a464-f3b27b6b2eba admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:27.595272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 898/1789] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:27.597671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.597817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.598048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.691666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:27.692963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.703869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fbf211-45a6-4dff-8dc8-39f871868cbb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:27.717119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:27.717755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.718023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.718302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.727342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 899/1790] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:27.727836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.728391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.733627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.764541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:27.769762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ae3af-825a-4ab2-8e21-6baa55d41032 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:27.795435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 892/1791] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:27.797729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.809708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.810098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.815136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f9aa1-9b17-4ec6-b346-3cd16b3e0925 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:27.837538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:27.845179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:27.845863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:27.846140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:27.846406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:27.852867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:27.858117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:27.881293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:27.884492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:27.928420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 893/1792] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:27.928420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.929324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 900/1793] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:27.935479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:27.977075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212050e-b7c0-45df-b1c5-c721aff8d90e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:28.010891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46be91ef-bd91-483f-a0fb-e129d4eca7b7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:28.026388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:28.026522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:28.028698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.029178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.029548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.030215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:28.030526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:28.031204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.032047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.033767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] PATH_INFO: `/v3/projects/28fbd0d59cf84477ba21a8b504172a04/users/ade125f6e988436cb726bcadaeacfecf/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.036875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:28.084092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 901/1794] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:28.085366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:28.107444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28fbd0d59cf84477ba21a8b504172a04, user_id=ade125f6e988436cb726bcadaeacfecf, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:28.121335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8733c1a-cd1e-4de5-a6d2-ed8ae813625c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:28.145435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e1b55d-1762-4e4d-8b85-e6f214bef65c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:28.162872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 894/1795] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:27 2022] PUT /identity/v3/projects/28fbd0d59cf84477ba21a8b504172a04/users/ade125f6e988436cb726bcadaeacfecf/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:28.173950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.174543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.175045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.305523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] MFA Rules not processed for user `ade125f6e988436cb726bcadaeacfecf`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:28.306538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:28.439871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:28.440460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.440745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.441029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.448553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:28.454792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:28.547481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 895/1796] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:28.548871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:28.581637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 902/1797] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:28 2022] POST /identity/v3/projects => generated 365 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:28.582546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:28.596692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c39d4-6c0d-45ee-b669-37f9945d4c35 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:28.625018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:28.916170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e23e9-ff75-4cf2-8cdf-e8ff42a6a293 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:28.927112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:28.927968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.933270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.933270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.933914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:28.942388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:28.945111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:28.946087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:28.947959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:28.948695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:28.955003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:28.958379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.017589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 896/1798] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:28 2022] POST /identity/v3/projects => generated 381 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:29.018805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.027296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 903/1799] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:28 2022] POST /identity/v3/users => generated 283 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:29.039055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.096166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09893d6-d3f8-480e-9b31-5209bbbada14 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.144198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:29.174952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ec602-7aee-4356-bd80-9f95a9801bd4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.188767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.189768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.190257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.190673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.193052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.205579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.245635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 897/1800] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:29 2022] POST /identity/v3/users => generated 297 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:29.249433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.270853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68129dd8-5f7f-4475-bf4e-07176b0a4cfe admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.286659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.287527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.287952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.288397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.289185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.291164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.316027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 898/1801] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:29.320058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.354765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b08683-96b9-4c12-887e-a2d547192206 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.361291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.362084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.362464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.362868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.363633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.374907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.375594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.375870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.376148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] PATH_INFO: `/v3/projects/e101ffa4451348db919d73f5c0be7476/users/abdd1c0a768c4549a50a2db72c4d3eb1/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.377720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.402716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e101ffa4451348db919d73f5c0be7476, user_id=abdd1c0a768c4549a50a2db72c4d3eb1, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.406286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.419713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 904/1802] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:29.424407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.438694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 899/1803] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:29 2022] PUT /identity/v3/projects/e101ffa4451348db919d73f5c0be7476/users/abdd1c0a768c4549a50a2db72c4d3eb1/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:29.443894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.463003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45700f-b3a6-4832-ae5a-7d81a59572f8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.689148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.689763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.690469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.691498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.692389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.694237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.732761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 905/1804] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:29.737821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.784339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f84a1-de85-4b46-bb91-ca89312ff0f7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.790306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823b35bd-3da4-436b-9f0b-8a69b8f84a2a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:29.814045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.815328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:29.816808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.824581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.826333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.828387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.829110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:29.829797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:29.830265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] PATH_INFO: `/v3/projects/046c941bab404e88b4b15bd09a15cbd4/users/0aebd02dac0f43e7be9024d5f76d9237/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:29.840630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.889112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 900/1805] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 446 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:29.900689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:29.923607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=046c941bab404e88b4b15bd09a15cbd4, user_id=0aebd02dac0f43e7be9024d5f76d9237, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:29.942097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0466c-df78-432a-a289-eb3fb729eab4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:29.980728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321bd51-2feb-4539-9832-1bfb46a55164 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:30.004882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 906/1806] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:29 2022] PUT /identity/v3/projects/046c941bab404e88b4b15bd09a15cbd4/users/0aebd02dac0f43e7be9024d5f76d9237/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:30.015763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:30.016335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:30.016785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:30.048111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:30.143416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] MFA Rules not processed for user `0aebd02dac0f43e7be9024d5f76d9237`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:30.166018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:30.280927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:30.281575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:30.281817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:30.282064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] PATH_INFO: `/v3/projects/e101ffa4451348db919d73f5c0be7476/users/abdd1c0a768c4549a50a2db72c4d3eb1/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:30.300761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e101ffa4451348db919d73f5c0be7476, user_id=abdd1c0a768c4549a50a2db72c4d3eb1, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:30.303432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b5b16-f1e5-48dd-9d51-1a444ddd591a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:30.337061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 901/1807] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:29 2022] PUT /identity/v3/projects/e101ffa4451348db919d73f5c0be7476/users/abdd1c0a768c4549a50a2db72c4d3eb1/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:30.347340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80183994-0769-4833-9bdd-7ec2cdf95825 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:30.347963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80183994-0769-4833-9bdd-7ec2cdf95825 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:30.348530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80183994-0769-4833-9bdd-7ec2cdf95825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:30.455675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-80183994-0769-4833-9bdd-7ec2cdf95825 None None] MFA Rules not processed for user `abdd1c0a768c4549a50a2db72c4d3eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:30.455879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 907/1808] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:30.459811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-80183994-0769-4833-9bdd-7ec2cdf95825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:30.465258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:30.503395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b311c-7ee0-4aa7-9378-1e37e288bbab None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:30.554115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:30.705242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:30.705971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:30.706549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:30.706653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 902/1809] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:30.707812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:30.708472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:30.710325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:30.710541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:30.710806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:30.717120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:30.791796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] MFA Rules not processed for user `49c772b0c2bb49a4ab4048d980bd20a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:30.793104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:30.817456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 908/1810] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:30 2022] POST /identity/v3/projects => generated 363 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:30.818399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:30.827093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:30.914285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 903/1811] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:30.914285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:30.932960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e6d2b-8abb-445a-a90c-ea02be730c62 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.013569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfa6ca-2a56-4592-8005-ac35e23014e1 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.025472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.026595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.026882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.027156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.095629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.098348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.110255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.110526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.111098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.118752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.126739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:31.169772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 904/1812] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:30 2022] POST /identity/v3/users => generated 288 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:31.173948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:31.196592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.197208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df63c0-cd00-42c5-93a9-ec632e466e31 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.199942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.209784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.212662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.213182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.214320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.216720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.220939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.228938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 909/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:31.258807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 905/1814] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:31.263263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:31.302908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce3612-2a34-448d-871e-bfbb62c817e2 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.324391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.327241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.329301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.329614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] PATH_INFO: `/v3/projects/65a1b841cd344da7880a2338bd0e89f5/users/9364a68de970441da459d16ef1121185/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.430392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a1b841cd344da7880a2338bd0e89f5, user_id=9364a68de970441da459d16ef1121185, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.436887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.488822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 910/1815] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:31 2022] PUT /identity/v3/projects/65a1b841cd344da7880a2338bd0e89f5/users/9364a68de970441da459d16ef1121185/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:31.490236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:31.502913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6719e43-90dc-46b9-8f9b-feadf8fffd67 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.696813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:31.760811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.761449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.761732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.762004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.762644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.764261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e917f55a-57b7-4932-bf3e-dafbd03a0321 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.810916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:31.823638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 906/1816] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:31.833341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c345ae4-32c5-4c4e-bd65-22c5a0be2fea admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:31.847294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:31.848466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.849317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.849988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] PATH_INFO: `/v3/projects/65a1b841cd344da7880a2338bd0e89f5/users/9364a68de970441da459d16ef1121185/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:31.856921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65a1b841cd344da7880a2338bd0e89f5, user_id=9364a68de970441da459d16ef1121185, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:31.865989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:31.917620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 911/1817] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:31 2022] PUT /identity/v3/projects/65a1b841cd344da7880a2338bd0e89f5/users/9364a68de970441da459d16ef1121185/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:31.922699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:31.922987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:31.923270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:32.020788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] MFA Rules not processed for user `9364a68de970441da459d16ef1121185`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:32.022063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:32.243030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 907/1818] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:32.262342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:32.265322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:32.502781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:32.522277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3a8b8-0762-427f-96d6-dbb5926fe6f2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:32.540970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:32.544038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:32.544381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:32.545971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:32.576567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:32.581159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:32.625319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 912/1819] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:32.751528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924ef53-01d8-4d61-bcd8-b5662592ffde None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:32.777245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdd1c0a768c4549a50a2db72c4d3eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e101ffa4451348db919d73f5c0be7476', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:32.777963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:32.778315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:32.778658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:32.779376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:32.779815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:32.780828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:32.783273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:32.795278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6a8af-294b-4add-a4eb-c09097e79e60 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:32.806892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 908/1820] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:32.811596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:32.816941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:32.817569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:32.817841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:32.818110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:32.838105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3008d2-0d7e-435c-8940-d2b48e32cc09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:32.875353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdd1c0a768c4549a50a2db72c4d3eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e101ffa4451348db919d73f5c0be7476', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:32.876835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:32.877125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:32.877400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:32.882504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:32.891930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:32.985754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 909/1821] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:32 2022] POST /identity/v3/projects => generated 354 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:32.990875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:33.054135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:33.059170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:33.090419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 913/1822] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:33.100794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:33.149680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6f371-bd58-4eda-916b-5184b98f57da tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:33.159121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdd1c0a768c4549a50a2db72c4d3eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e101ffa4451348db919d73f5c0be7476', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:33.160778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:33.160778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:33.161126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:33.162656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:33.165179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:33.166911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:33.194382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 910/1823] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:33.277603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:33.279257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:33.401565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:33.431190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:33.447328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da329ab2-f905-4585-bc5a-6f6fed4f2e92 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:33.468345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abdd1c0a768c4549a50a2db72c4d3eb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e101ffa4451348db919d73f5c0be7476', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:33.471485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:33.471485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:33.471677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] PATH_INFO: `/v3/projects/06b6655c8a1d4a67ac53d3312462d4be` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:33.479664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06b6655c8a1d4a67ac53d3312462d4be)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:33.486751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:33.529035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270dc50-4aa1-4963-a294-ec076efbdb30 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:33.725654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:33.726380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:33.727775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:33.728130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] PATH_INFO: `/v3/users/4383d2dfd0224f50be0569ffdfa6e3b7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:33.732595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b6655c8a1d4a67ac53d3312462d4be', 'request_id': 'req-a0ba58b1-b493-4fb9-a616-949261e2ab09'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.737865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4383d2dfd0224f50be0569ffdfa6e3b7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:33.740363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:33.757307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b6655c8a1d4a67ac53d3312462d4be', 'request_id': 'req-a0ba58b1-b493-4fb9-a616-949261e2ab09'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.782900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b6655c8a1d4a67ac53d3312462d4be', 'request_id': 'req-a0ba58b1-b493-4fb9-a616-949261e2ab09'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.888224 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] The token cache is being invalidate because project 06b6655c8a1d4a67ac53d3312462d4be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:33.888932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.891526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a0ba58b1-b493-4fb9-a616-949261e2ab09 tempest-VolumeQuotaClassesTest-2134471795 tempest-VolumeQuotaClassesTest-2134471795-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.907390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4383d2dfd0224f50be0569ffdfa6e3b7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.913101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 911/1824] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:33 2022] DELETE /identity/v3/projects/06b6655c8a1d4a67ac53d3312462d4be => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:33.935338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4383d2dfd0224f50be0569ffdfa6e3b7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.937705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4383d2dfd0224f50be0569ffdfa6e3b7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:33.971808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4383d2dfd0224f50be0569ffdfa6e3b7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.041508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4383d2dfd0224f50be0569ffdfa6e3b7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.057582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 914/1825] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:33 2022] DELETE /identity/v3/users/4383d2dfd0224f50be0569ffdfa6e3b7 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:34.062469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:34.062821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:34.063144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:34.126613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:34.126613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:34.338343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 912/1826] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:34.380730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:34.389000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:34.546430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:34.556267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:34.580469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f41c70-14e7-4d4f-8feb-4e3c0a8ff52b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:34.600699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:34.601609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:34.606683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:34.607258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] PATH_INFO: `/v3/users/ade125f6e988436cb726bcadaeacfecf` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:34.613343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade125f6e988436cb726bcadaeacfecf)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:34.621448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e58f3f-1380-43d5-bd7d-a655bc52fe01 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:34.622965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:34.629419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:34.630783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:34.631261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:34.631648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:34.653052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:34.655230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e3526-e053-4f71-9943-3b266b398dc8 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:34.672935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 915/1827] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:34.769797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade125f6e988436cb726bcadaeacfecf'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.776115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:34.791082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade125f6e988436cb726bcadaeacfecf'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.817645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade125f6e988436cb726bcadaeacfecf'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.819036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade125f6e988436cb726bcadaeacfecf'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.828064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade125f6e988436cb726bcadaeacfecf'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:34.839413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 913/1828] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:34 2022] DELETE /identity/v3/users/ade125f6e988436cb726bcadaeacfecf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:34.841295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c75-fc9e-4a9b-9d43-dd839628cafe None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:34.841578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c75-fc9e-4a9b-9d43-dd839628cafe None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:34.841851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d354c75-fc9e-4a9b-9d43-dd839628cafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:34.860353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:34.902501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1d354c75-fc9e-4a9b-9d43-dd839628cafe None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:34.905636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1d354c75-fc9e-4a9b-9d43-dd839628cafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:35.079008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 916/1829] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:35.158535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:35.162248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:35.376060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfabc5a-edc1-4854-b19b-695aa1aa3ca2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:35.392427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:35.393185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:35.393964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:35.395851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:35.430195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:35.433715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6667f-96a9-4969-a1f4-8f75af6c687f service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:35.476122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 914/1830] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:35.874065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:35.875744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:35.876220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:35.878044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 917/1831] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:29:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:35.894594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:35.895999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:35.896650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:35.964898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:35.966276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:36.094019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:36.098065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:36.150431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 915/1832] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 22 12:29:36.151833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05187be-8085-47ad-8614-e5410d44de86 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:36.165060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:36.167574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:36.167574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:36.167574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] PATH_INFO: `/v3/projects/37bb9db0546f4906806214f1f1131fdf` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:36.168464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37bb9db0546f4906806214f1f1131fdf)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:36.170688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:36.233609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37bb9db0546f4906806214f1f1131fdf', 'request_id': 'req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.247813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37bb9db0546f4906806214f1f1131fdf', 'request_id': 'req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.260681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37bb9db0546f4906806214f1f1131fdf', 'request_id': 'req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.272645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] The token cache is being invalidate because project 37bb9db0546f4906806214f1f1131fdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:36.272936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.273611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-40ec8c31-7bfc-4e14-b570-9d0a9f06b7e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.275474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 918/1833] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:36 2022] DELETE /identity/v3/projects/37bb9db0546f4906806214f1f1131fdf => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:36.279309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:36.297699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d816df-7f41-4c35-a98f-1ff9c341b431 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:36.394290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:36.395322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:36.395916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:36.396041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] PATH_INFO: `/v3/users/184d77907d9a402c8526c0f758bf9b63` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:36.397533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=184d77907d9a402c8526c0f758bf9b63)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:36.400580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:36.403584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:36.427344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1b1d-badb-4dcc-bb8b-d31df2dee44d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:36.427619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1b1d-badb-4dcc-bb8b-d31df2dee44d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:36.427951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb1b1d-badb-4dcc-bb8b-d31df2dee44d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:36.429335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 919/1834] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:36.437361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:36.437620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:36.437854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:36.491664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:36.494470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:36.502276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184d77907d9a402c8526c0f758bf9b63'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.513296 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184d77907d9a402c8526c0f758bf9b63'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.559580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184d77907d9a402c8526c0f758bf9b63'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.561668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184d77907d9a402c8526c0f758bf9b63'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.592860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184d77907d9a402c8526c0f758bf9b63'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:36.606006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 916/1835] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:36 2022] DELETE /identity/v3/users/184d77907d9a402c8526c0f758bf9b63 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:36.712148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 920/1836] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:37.037995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:37.041074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:37.242234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2573ba9-295a-4b82-89a1-d5f0fff26ccf admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:37.249025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:37.249727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:37.250073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:37.251019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:37.252921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:37.269883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:37.271928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:37.284077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 917/1837] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:37.512433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:37.517530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:37.543782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63948c-5cd4-4cc1-adab-e05898780d6a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:37.557845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:37.558927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:37.559412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:37.560003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:37.577084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:37.590288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:37.607082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 921/1838] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:37.888348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:37.889944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:37.927487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:37.929434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:37.944427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:38.036573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2221ad9-7c08-4dc8-9714-95f63a079c7f service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.046281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.046903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.047186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.047459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] PATH_INFO: `/v3/projects/28fbd0d59cf84477ba21a8b504172a04` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.048615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28fbd0d59cf84477ba21a8b504172a04)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.051487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.061631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:38.117770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdd450-c7f6-4242-9ee5-c88333592079 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.164582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28fbd0d59cf84477ba21a8b504172a04', 'request_id': 'req-f9a71a62-dff4-4021-830c-320278123311'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.182549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28fbd0d59cf84477ba21a8b504172a04', 'request_id': 'req-f9a71a62-dff4-4021-830c-320278123311'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.191160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28fbd0d59cf84477ba21a8b504172a04', 'request_id': 'req-f9a71a62-dff4-4021-830c-320278123311'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.216580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] The token cache is being invalidate because project 28fbd0d59cf84477ba21a8b504172a04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:38.216580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.216580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.219417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.219709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 918/1839] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:37 2022] DELETE /identity/v3/projects/28fbd0d59cf84477ba21a8b504172a04 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:38.220354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.220733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.221066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] PATH_INFO: `/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.222839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=155bf88ed03e48d7a2f9323ad7dd24e1)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.223993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.225405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:38.226113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.256021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a71a62-dff4-4021-830c-320278123311 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.270308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.271628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.272501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.273152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] PATH_INFO: `/v3/users/abdd1c0a768c4549a50a2db72c4d3eb1` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.277501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdd1c0a768c4549a50a2db72c4d3eb1)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.279637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.335344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155bf88ed03e48d7a2f9323ad7dd24e1', 'request_id': 'req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.354643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155bf88ed03e48d7a2f9323ad7dd24e1', 'request_id': 'req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.364453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155bf88ed03e48d7a2f9323ad7dd24e1', 'request_id': 'req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.377154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdd1c0a768c4549a50a2db72c4d3eb1'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.386417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] The token cache is being invalidate because project 155bf88ed03e48d7a2f9323ad7dd24e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:38.386953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.387648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-be65a9f8-7af6-4748-9e51-da8c24ed8eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.388452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdd1c0a768c4549a50a2db72c4d3eb1'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.392390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 922/1840] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:37 2022] DELETE /identity/v3/projects/155bf88ed03e48d7a2f9323ad7dd24e1 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:38.394672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.394957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.395302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.405169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdd1c0a768c4549a50a2db72c4d3eb1'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.406342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdd1c0a768c4549a50a2db72c4d3eb1'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.422181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2fd2311d-0c20-41b4-99b4-ae66f8721e7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdd1c0a768c4549a50a2db72c4d3eb1'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.422883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:38.424901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:38.429697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 919/1841] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:38 2022] DELETE /identity/v3/users/abdd1c0a768c4549a50a2db72c4d3eb1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:38.492564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:38.524148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 923/1842] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:38.532242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.532639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.533047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.566133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:38.567354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:38.686412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 920/1843] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:38.687303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.703754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57a717-9c5c-49a3-bed9-f3f134b59aa6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.710508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.711138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.711410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.711925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.712826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:38.713307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.715188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.731873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 924/1844] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:38.750590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.761358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab7cc-74fd-4589-b5fa-d1972015a474 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.766156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.766780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.767090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.767361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.767989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.770583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.775310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:38.804403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 921/1845] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:38 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:38.809294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.821284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041d703-d654-477a-919a-89f4b4e50708 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.837577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.841226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:38.921168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.921881 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.922313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.922646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.923339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.925330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.926888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a663-7dfe-4d4b-94e9-d5d84b12dba4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.931504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.932195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.932560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.932893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] PATH_INFO: `/v3/projects/e101ffa4451348db919d73f5c0be7476` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.934084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e101ffa4451348db919d73f5c0be7476)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.936048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.948134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 925/1846] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:38 2022] POST /identity/v3/users => generated 284 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:38.952209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.962233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95371d09-4008-4dea-aa00-5dfd707a75d7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:38.967308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:38.967904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:38.968176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:38.968475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:38.969306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:38.971042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:38.985389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 926/1847] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:38.987077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e101ffa4451348db919d73f5c0be7476', 'request_id': 'req-affd9c83-53c7-4346-8899-b5d337e15d2d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:38.987189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:38.995048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e101ffa4451348db919d73f5c0be7476', 'request_id': 'req-affd9c83-53c7-4346-8899-b5d337e15d2d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:39.001455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514960-86b5-458d-99f2-6d958e9a8dc0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.001858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e101ffa4451348db919d73f5c0be7476', 'request_id': 'req-affd9c83-53c7-4346-8899-b5d337e15d2d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:39.014605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] The token cache is being invalidate because project e101ffa4451348db919d73f5c0be7476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:39.015239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:39.015957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-affd9c83-53c7-4346-8899-b5d337e15d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:39.020362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 922/1848] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:38 2022] DELETE /identity/v3/projects/e101ffa4451348db919d73f5c0be7476 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:39.023816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392faedc-561c-465b-b780-959d04ad6d26 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.024160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392faedc-561c-465b-b780-959d04ad6d26 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.024757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392faedc-561c-465b-b780-959d04ad6d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.045947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-392faedc-561c-465b-b780-959d04ad6d26 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:39.047293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-392faedc-561c-465b-b780-959d04ad6d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.071890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.076221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.076674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.076961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.077617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.079348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.092976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 927/1849] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:39.094739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.096696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.109375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da0506-658b-46af-8029-18b447232eb6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.114554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 923/1850] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:39.114746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.115400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.115681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.115981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] PATH_INFO: `/v3/projects/3a6ec86204064d948614651cb8ff86d5/users/9a9a58b8ce03444c88a2e73057c36c42/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.122602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.122892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.123171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.132505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6ec86204064d948614651cb8ff86d5, user_id=9a9a58b8ce03444c88a2e73057c36c42, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.135163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114440f4-224e-439f-9118-79bcd31d908e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.156365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 928/1851] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:39 2022] PUT /identity/v3/projects/3a6ec86204064d948614651cb8ff86d5/users/9a9a58b8ce03444c88a2e73057c36c42/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:39.156541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:39.156541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.160514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.160820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.161086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.167451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:39.191737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] MFA Rules not processed for user `9a9a58b8ce03444c88a2e73057c36c42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:39.192982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.201053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:39.221057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 924/1852] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:39.225227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.234124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743867e-c637-439c-acd1-415e8a0e5a99 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.239226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.239808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.240085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.240391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.241050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:39.241350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.242964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.262525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 925/1853] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:29:39.263482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.263586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 929/1854] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:39.268574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.273343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29762a61-efef-495f-b2a8-912ee589f84f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.279084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.279688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.279966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.280238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.280530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f1089-c3a7-433b-aed8-e72dcee2e0a9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.280910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.282822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.285797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.286387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.286665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.286934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.287557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.289465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.313358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 926/1855] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:39.314379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.317112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 930/1856] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:39.321877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.378975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b182e32-9f95-4f9c-9cb6-1e1d4efc443d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.383729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.384348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.384632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.384816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dba46cf-fe43-4dc2-bcba-2e5a6beb2ab2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.384970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.385518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.387426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.389627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.390219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.390484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.390760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.391368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.393301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.416100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 927/1857] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:39.416849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 931/1858] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:39.417047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.420541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.427468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af518-32e2-4710-b103-9bb383a56f42 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.431748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3ba37-6c97-4e19-90a0-a778f7b49a3d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.432243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.432871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.433153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.433425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.434110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.435648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.436789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.437384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.437660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.438095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.438561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.440180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.447408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 928/1859] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:39.451184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 932/1860] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:39.452611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.455506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.466123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaf264-ec0d-48ff-b6e0-2a6dc081b5b7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.473465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.474233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.474625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.474994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] PATH_INFO: `/v3/projects/2c10403b098a475284d069f50d345043/users/97175f738b3c4f7b9b53537f7bb62e47/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.481842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf8e09-4311-46df-b539-ac9e5a362d81 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.492194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.493132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.493266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.493422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.494219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.495798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.498702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c10403b098a475284d069f50d345043, user_id=97175f738b3c4f7b9b53537f7bb62e47, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.502346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.507957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 929/1861] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:39.514320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.518543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 933/1862] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:39 2022] PUT /identity/v3/projects/2c10403b098a475284d069f50d345043/users/97175f738b3c4f7b9b53537f7bb62e47/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:39.522509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.526577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eeae47-283a-4990-8292-84e4bf5cba66 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.569616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:39.571393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:39.583629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.584288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.584592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.584869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] PATH_INFO: `/v3/projects/9d987f3d00ae480996f8e59e40661f5e/users/d905c0df45434d32af174332e6dc1042/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.588581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485190f-756a-4aa2-aa5b-3a9257695eeb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.593340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.593935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.594319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.594592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.595258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.597480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.601647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d987f3d00ae480996f8e59e40661f5e, user_id=d905c0df45434d32af174332e6dc1042, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.605302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f4e81-fec0-444f-b468-e08087568c43 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.608981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 934/1863] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:39.613641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.619536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 930/1864] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:39 2022] PUT /identity/v3/projects/9d987f3d00ae480996f8e59e40661f5e/users/d905c0df45434d32af174332e6dc1042/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:39.623594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1f14b-a8d0-4031-8682-db4248ca04b8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.629522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.629856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.630131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.664150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] MFA Rules not processed for user `d905c0df45434d32af174332e6dc1042`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:39.665504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.682959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.683701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.684045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.684561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] PATH_INFO: `/v3/projects/2c10403b098a475284d069f50d345043/users/97175f738b3c4f7b9b53537f7bb62e47/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.686865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c10403b098a475284d069f50d345043, user_id=97175f738b3c4f7b9b53537f7bb62e47, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.690317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd4f27-fce9-462d-af16-5b89224c14b3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.708381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 935/1865] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:39 2022] PUT /identity/v3/projects/2c10403b098a475284d069f50d345043/users/97175f738b3c4f7b9b53537f7bb62e47/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:39.716074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.716475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.716838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.750336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] MFA Rules not processed for user `97175f738b3c4f7b9b53537f7bb62e47`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:39.751763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.760001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 931/1866] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:39.764532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.776426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894366c-8840-4f88-ab10-0911c2a5cdd3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:39.849034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:39.849727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:39.850096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:39.850439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:39.851208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:39.853900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 936/1867] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:39.854121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:39.872747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:39.872747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:39.923517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 932/1868] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:39.926226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.004375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:40.012359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f744-c5a9-4e58-924a-70ae75f6f19f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.023814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d8f3d-0207-4a57-997b-a7542b9fdc9a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.033006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:40.034920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.035548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.035835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.036128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.037659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.039666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.064704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 933/1869] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:39 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:40.068465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.078706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17668153-02a5-4711-b6a0-8678b8bf1c2f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.083889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.084536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.084836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.085095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.085739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.087415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.092222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.092894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.093163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.093586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.099130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 934/1870] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:40.103671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.113329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c308c6a-01d6-489c-b250-9e7a1b1ac145 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.118122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.118763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.119041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.119320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] PATH_INFO: `/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d/users/7f95452101dd4d86b47311dd993c31ff/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.134689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c0b1d3c7e64f1fb02ec69c077cef7d, user_id=7f95452101dd4d86b47311dd993c31ff, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.137369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.151863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 935/1871] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:40 2022] PUT /identity/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d/users/7f95452101dd4d86b47311dd993c31ff/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:40.155965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.157408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.158856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.170940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 937/1872] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:40.233801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e89c79-eae0-491a-b476-18171477893e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.239741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.240400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.240684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.240957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.241609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.243355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.258678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 936/1873] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:40.262973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.278487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10c826-8a6f-4864-a594-8a6a14943006 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.293038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.294370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.294832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.297580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.300689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] PATH_INFO: `/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d/users/7f95452101dd4d86b47311dd993c31ff/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.313399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c0b1d3c7e64f1fb02ec69c077cef7d, user_id=7f95452101dd4d86b47311dd993c31ff, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.320778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11c1b7-c078-4e22-9500-cc9b9ab334f7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.334426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 938/1874] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:40 2022] PUT /identity/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d/users/7f95452101dd4d86b47311dd993c31ff/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:40.345381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.347703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.348092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.411323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] MFA Rules not processed for user `7f95452101dd4d86b47311dd993c31ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:40.412619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:40.413926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f60b2-b089-4cfb-837e-4d77e3c9d8f7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.481983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.482612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.482893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.483167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.490143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:40.504407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 939/1875] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:40.563378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.564953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.577504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:40.579672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:40.584097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 937/1876] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:40.594253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae29a70-8287-4c52-b106-45e1d75067a3 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:40.602637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:40.603344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:40.603865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:40.604288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:40.618725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:40.621221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63061304-fefd-4296-8f3d-993cdfacc6bd service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:40.634737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 940/1877] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:41.639654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:41.715460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0ca778-15ea-4f84-86ee-28eb98042d32 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:41.720473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:41.721059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:41.721370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:41.721606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] PATH_INFO: `/v3/users/d905c0df45434d32af174332e6dc1042` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:41.722835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d905c0df45434d32af174332e6dc1042)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:41.724831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:41.773534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd905c0df45434d32af174332e6dc1042'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:41.781117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd905c0df45434d32af174332e6dc1042'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:41.796682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd905c0df45434d32af174332e6dc1042'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:41.797547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd905c0df45434d32af174332e6dc1042'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:41.807565 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd905c0df45434d32af174332e6dc1042'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:41.816085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 938/1878] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:41 2022] DELETE /identity/v3/users/d905c0df45434d32af174332e6dc1042 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:41.820273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:41.820792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:41.820883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:41.842358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:41.843835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:41.862515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:41.920037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 941/1879] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:41.930657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:41.932188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:41.972863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:42.004403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e15f50-436e-402a-8a9c-250f7d46d029 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:42.008185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:42.008633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:42.008980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:42.009228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:42.025260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:42.026930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:42.056372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 939/1880] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:42.454187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:42.460371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:42.515687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f8d3f-b78e-455f-b8ac-cf93071975b2 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:42.530311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:42.531009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:42.531290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:42.531564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] PATH_INFO: `/v3/projects/9d987f3d00ae480996f8e59e40661f5e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:42.534800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d987f3d00ae480996f8e59e40661f5e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:42.539705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:42.628309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d987f3d00ae480996f8e59e40661f5e', 'request_id': 'req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.649899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d987f3d00ae480996f8e59e40661f5e', 'request_id': 'req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.660955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d987f3d00ae480996f8e59e40661f5e', 'request_id': 'req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.708121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] The token cache is being invalidate because project 9d987f3d00ae480996f8e59e40661f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:42.708440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.711012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.716263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 942/1881] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:42 2022] DELETE /identity/v3/projects/9d987f3d00ae480996f8e59e40661f5e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:42.723063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:42.746589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7b5d6-169a-4a32-989f-89ef18890784 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:42.821816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:42.823209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:42.823566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:42.823912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] PATH_INFO: `/v3/users/7f95452101dd4d86b47311dd993c31ff` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:42.825246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f95452101dd4d86b47311dd993c31ff)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:42.827306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:42.936662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f95452101dd4d86b47311dd993c31ff'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.951087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f95452101dd4d86b47311dd993c31ff'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.987780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f95452101dd4d86b47311dd993c31ff'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.988698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f95452101dd4d86b47311dd993c31ff'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:42.998755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f95452101dd4d86b47311dd993c31ff'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:43.025155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 940/1882] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:42 2022] DELETE /identity/v3/users/7f95452101dd4d86b47311dd993c31ff => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:44.510291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:44.512431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:44.533601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8632c-8ba8-4ba9-b54b-8adaf5ce6ff2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:44.655366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:44.656116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:44.656318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:44.656650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] PATH_INFO: `/v3/users/0aebd02dac0f43e7be9024d5f76d9237` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:44.663717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aebd02dac0f43e7be9024d5f76d9237)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:44.666698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:44.720770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:44.722375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:44.741752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30261773-2901-4786-9f50-d265e4c09147 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:44.757065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:44.758729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:44.759203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:44.759685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] PATH_INFO: `/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:44.761500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c0b1d3c7e64f1fb02ec69c077cef7d)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:44.765481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:44.777774 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aebd02dac0f43e7be9024d5f76d9237'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.784634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:44.789265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aebd02dac0f43e7be9024d5f76d9237'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.793288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aebd02dac0f43e7be9024d5f76d9237'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.804535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aebd02dac0f43e7be9024d5f76d9237'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.840354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:44.851044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c0b1d3c7e64f1fb02ec69c077cef7d', 'request_id': 'req-702b566d-e0cb-4e3b-934c-8be15b230dc4'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.862795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-941fe80a-3dd4-4040-8eae-39ed5c914d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aebd02dac0f43e7be9024d5f76d9237'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.901610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c0b1d3c7e64f1fb02ec69c077cef7d', 'request_id': 'req-702b566d-e0cb-4e3b-934c-8be15b230dc4'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.906856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 943/1883] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:44 2022] DELETE /identity/v3/users/0aebd02dac0f43e7be9024d5f76d9237 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:44.910927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784479-4fce-4c70-8487-86581320defb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:44.912136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784479-4fce-4c70-8487-86581320defb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:44.913660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec784479-4fce-4c70-8487-86581320defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:44.922210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c0b1d3c7e64f1fb02ec69c077cef7d', 'request_id': 'req-702b566d-e0cb-4e3b-934c-8be15b230dc4'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.961588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] The token cache is being invalidate because project 20c0b1d3c7e64f1fb02ec69c077cef7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:44.962062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.963146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-702b566d-e0cb-4e3b-934c-8be15b230dc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:44.965592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 941/1884] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:44 2022] DELETE /identity/v3/projects/20c0b1d3c7e64f1fb02ec69c077cef7d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:44.970218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c7471-82e7-4064-8b4b-a661a8d63bff None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:44.970700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c7471-82e7-4064-8b4b-a661a8d63bff None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:44.973986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44c7471-82e7-4064-8b4b-a661a8d63bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:44.977480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ec784479-4fce-4c70-8487-86581320defb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:44.981235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ec784479-4fce-4c70-8487-86581320defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.037584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f44c7471-82e7-4064-8b4b-a661a8d63bff None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:45.039059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f44c7471-82e7-4064-8b4b-a661a8d63bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.110677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 944/1885] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:45.113720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.114195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.114665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.117916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 945/1886] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:29:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:29:45.151370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.155275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.196763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 942/1887] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:45.198119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.198506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.198755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.249517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:45.250774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.266428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ec1a0-3bf4-444f-9834-135fcdada3fa None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.273878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.275650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.276049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.276477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.309345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.311211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94133916-2af3-419e-a053-e2a2e5b43087 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.340010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 946/1888] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:45.345263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.345645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.346670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.378073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:45.379873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.391497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 947/1889] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:45.396693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.408352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e371d-b65b-4cd7-a0f3-c29f1e3df529 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.410445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 943/1890] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:45.415644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.416903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.417293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.417629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.418356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:45.427211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.428160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.428916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.430026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.454517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d29d1-b500-4429-b15d-f741b8b3eca8 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.469034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.469823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.470109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.470380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.485335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.488029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.501425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 948/1891] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:29:45.506433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.523264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 944/1892] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:45.533329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29a72b-9e38-4732-9788-e306fa404f54 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.540381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.541691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.541975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.542543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.543194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.546114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.606187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 949/1893] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:45 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:45.612271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.631396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de687a-6e5a-450a-83c3-9387354bd145 service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.718108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:45.744785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.745528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.746051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.746434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.747463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.749477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.773579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 945/1894] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:45 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:45.777462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.787933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccb0e9-3e79-46f4-909d-6f39ea13ed4a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.792624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.793246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.793524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.793792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.794419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.796016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.806844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 950/1895] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:45.810637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.821917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83938de1-9da6-446c-8179-1e0b2d01bf49 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.826743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.827452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.827841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.828124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.828810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.830428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.842517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 946/1896] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:45.846400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.861754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e8a1ec-ffd0-444a-9430-28dfa08352b6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.883992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:45.884345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.884625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.884894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] PATH_INFO: `/v3/projects/2265e4f50bd34b37a0144856670df4cc/users/1f45d4681cac4045a2a856044e636ecc/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.898731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:45.902091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2265e4f50bd34b37a0144856670df4cc, user_id=1f45d4681cac4045a2a856044e636ecc, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:45.906104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746fdc-65ef-45f0-932a-cbf465963b80 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:45.929903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 951/1897] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:45 2022] PUT /identity/v3/projects/2265e4f50bd34b37a0144856670df4cc/users/1f45d4681cac4045a2a856044e636ecc/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:45.930186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:45.930186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:45.932260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:45.932579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:45.932855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:45.939077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0dbad6-e65b-4419-bb64-4315ac79398d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:45.970823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] MFA Rules not processed for user `1f45d4681cac4045a2a856044e636ecc`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:45.972095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.001321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:46.001943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:46.002217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:46.002483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] PATH_INFO: `/v3/projects/046c941bab404e88b4b15bd09a15cbd4` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:46.004126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] RBAC: Authorizing `identity:delete_project(project_id=046c941bab404e88b4b15bd09a15cbd4)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:46.006043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:46.041302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 952/1898] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:46.054376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:46.056014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.056392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '046c941bab404e88b4b15bd09a15cbd4', 'request_id': 'req-e4f87604-db4d-496d-83c7-14501ff24420'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.067685 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '046c941bab404e88b4b15bd09a15cbd4', 'request_id': 'req-e4f87604-db4d-496d-83c7-14501ff24420'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.074024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '046c941bab404e88b4b15bd09a15cbd4', 'request_id': 'req-e4f87604-db4d-496d-83c7-14501ff24420'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.087035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] The token cache is being invalidate because project 046c941bab404e88b4b15bd09a15cbd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:46.087318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.087968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.090108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 947/1899] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:45 2022] DELETE /identity/v3/projects/046c941bab404e88b4b15bd09a15cbd4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:46.094227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:46.095805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.179785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0c346-b997-4b3d-a780-d6f31efd8d66 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:46.191342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f87604-db4d-496d-83c7-14501ff24420 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:46.197089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:46.197808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:46.198091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:46.198364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] PATH_INFO: `/v3/users/9364a68de970441da459d16ef1121185` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:46.199575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9364a68de970441da459d16ef1121185)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:46.201525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:46.203382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:46.252360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:46.253274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:46.254914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:46.254914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:46.256744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.285541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9364a68de970441da459d16ef1121185'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.302100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9364a68de970441da459d16ef1121185'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.318689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:46.331131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9364a68de970441da459d16ef1121185'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.331959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9364a68de970441da459d16ef1121185'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.344644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9364a68de970441da459d16ef1121185'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.357049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 948/1900] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:46 2022] DELETE /identity/v3/users/9364a68de970441da459d16ef1121185 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:46.358371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:46.361761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.383026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:46.384778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cec88-010f-4242-8ea2-e67a095452a6 service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:46.402665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 953/1901] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:29:46.494412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c519c6f-4095-4566-a504-6a5d7a396a9f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:46.512832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:46.513487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:46.513763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:46.514350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] PATH_INFO: `/v3/users/9a9a58b8ce03444c88a2e73057c36c42` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:46.516935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a9a58b8ce03444c88a2e73057c36c42)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:46.519490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:46.621608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a9a58b8ce03444c88a2e73057c36c42'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.634229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a9a58b8ce03444c88a2e73057c36c42'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.650790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a9a58b8ce03444c88a2e73057c36c42'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.652706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a9a58b8ce03444c88a2e73057c36c42'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.665117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a9a58b8ce03444c88a2e73057c36c42'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:46.668589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:46.688560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 949/1902] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:46 2022] DELETE /identity/v3/users/9a9a58b8ce03444c88a2e73057c36c42 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:46.693692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:46.694182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:46.694330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:46.728659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:46.729880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:46.847946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 954/1903] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:46.900172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:46.902366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:47.038146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce5a2c4-b58d-4eb9-9ff6-e3908f5513f5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:47.046247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:47.047044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:47.047639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:47.048878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:47.078756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:47.081840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd507b5-3c48-4bcd-839f-8b49f3e36a22 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:47.105275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 950/1904] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:47.816153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:47.819212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:47.836703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa76091e-c6d5-4534-8c90-6deaff489827 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:47.842444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:47.843453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:47.843910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:47.844385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] PATH_INFO: `/v3/projects/65a1b841cd344da7880a2338bd0e89f5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:47.845826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65a1b841cd344da7880a2338bd0e89f5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:47.847989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:47.965923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65a1b841cd344da7880a2338bd0e89f5', 'request_id': 'req-2fb5a527-1838-46ef-911a-24f985a12c34'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.011725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65a1b841cd344da7880a2338bd0e89f5', 'request_id': 'req-2fb5a527-1838-46ef-911a-24f985a12c34'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.020586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65a1b841cd344da7880a2338bd0e89f5', 'request_id': 'req-2fb5a527-1838-46ef-911a-24f985a12c34'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.049267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] The token cache is being invalidate because project 65a1b841cd344da7880a2338bd0e89f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:48.050215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.051099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.053046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 955/1905] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:47 2022] DELETE /identity/v3/projects/65a1b841cd344da7880a2338bd0e89f5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:48.058317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36fe61-fd70-49c0-9e49-83b071c7d8b8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.059111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36fe61-fd70-49c0-9e49-83b071c7d8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.059111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36fe61-fd70-49c0-9e49-83b071c7d8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.082169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2f36fe61-fd70-49c0-9e49-83b071c7d8b8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:48.083532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f36fe61-fd70-49c0-9e49-83b071c7d8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.085290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.086791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.173478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:48.180089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5a527-1838-46ef-911a-24f985a12c34 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.184061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 951/1906] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:48.186824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.187469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.187747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.188020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] PATH_INFO: `/v3/projects/3a6ec86204064d948614651cb8ff86d5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.189234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6ec86204064d948614651cb8ff86d5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.191467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.191686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.191972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.192246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.212526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:48.213786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.223982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 952/1907] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:48.228816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.238882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832aeca-fe7e-4142-af9d-794b019e9cfd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.248196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6ec86204064d948614651cb8ff86d5', 'request_id': 'req-06e45504-37cb-4032-93e2-eb57d7618c88'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.256765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6ec86204064d948614651cb8ff86d5', 'request_id': 'req-06e45504-37cb-4032-93e2-eb57d7618c88'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.268190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6ec86204064d948614651cb8ff86d5', 'request_id': 'req-06e45504-37cb-4032-93e2-eb57d7618c88'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.291661 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] The token cache is being invalidate because project 3a6ec86204064d948614651cb8ff86d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:48.292196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.293198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.295468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 956/1908] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:48 2022] DELETE /identity/v3/projects/3a6ec86204064d948614651cb8ff86d5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:48.299014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.300551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.308105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.308738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.309013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.309282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.309915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:48.310214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.311783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.324341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 953/1909] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:29:48.328063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.332453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.343373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca3fc3-bccd-40ca-a4c1-7c89dac5d205 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.349043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.349043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.349043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.351428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.351428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.352928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.395339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 954/1910] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:48.399042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.417928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:48.434666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e45504-37cb-4032-93e2-eb57d7618c88 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.501306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa03950-1df8-4f26-93b0-4eff3cdcd366 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.505485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.506134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.506410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.506681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] PATH_INFO: `/v3/users/97175f738b3c4f7b9b53537f7bb62e47` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.507809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] RBAC: Authorizing `identity:delete_user(user_id=97175f738b3c4f7b9b53537f7bb62e47)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.509790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.511701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.512976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.513246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.513522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.514186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.516220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.552168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 955/1911] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:48.560014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.570814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab2f3-78b4-41bc-8611-902b75b6f61f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.576039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.576769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.577068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.577827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.577995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.579592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.591496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:48.596377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 956/1912] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:48.598181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.603911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97175f738b3c4f7b9b53537f7bb62e47'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.612697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b47b16-aee4-419b-acbf-e8389855ac04 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.615041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97175f738b3c4f7b9b53537f7bb62e47'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.616200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97175f738b3c4f7b9b53537f7bb62e47'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.618983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.619815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.620091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.620392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.621028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.622638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.624842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97175f738b3c4f7b9b53537f7bb62e47'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.634018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 957/1913] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:48.637913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fc684863-eca2-450d-a20d-06e5983f286f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97175f738b3c4f7b9b53537f7bb62e47'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:48.638036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.639696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.644416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 957/1914] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:48 2022] DELETE /identity/v3/users/97175f738b3c4f7b9b53537f7bb62e47 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:48.652642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceb04f-93aa-45c0-8b58-7e2aa137f335 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.713858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.714832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.715231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.715687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] PATH_INFO: `/v3/projects/6992f30c6d4c46f78ad0b0a15418809e/users/f3f6a0aaf4e44abbaace037584d70972/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.737176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6992f30c6d4c46f78ad0b0a15418809e, user_id=f3f6a0aaf4e44abbaace037584d70972, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.740162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:48.757374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 958/1915] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:48 2022] PUT /identity/v3/projects/6992f30c6d4c46f78ad0b0a15418809e/users/f3f6a0aaf4e44abbaace037584d70972/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:48.769108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.769467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.769806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.806573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] MFA Rules not processed for user `f3f6a0aaf4e44abbaace037584d70972`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:48.808273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:48.886866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 958/1916] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:48.891426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:48.955451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:48.958805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf09d-cb1b-4df1-a800-b271afadc581 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:48.965198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:48.966000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:48.966338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:48.966640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:48.967267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:48.969239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.012838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 959/1917] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:48 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:49.017852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.029360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469c96e-85de-49b2-a2a1-1261f1362971 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.108423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.109137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.109414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.109754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.110451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.112565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.113995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.116362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.132479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f88cfa-9cd4-4b62-b1d1-20232b6fe3d4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.137944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.138603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.138884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.139217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] PATH_INFO: `/v3/projects/2c10403b098a475284d069f50d345043` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.141111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c10403b098a475284d069f50d345043)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.142439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 959/1918] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:49.146395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.146944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.156603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352d9c-2fa5-4fb1-9e3c-c8210921119c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.162553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.163053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.163456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.163729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.164526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.166107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.177034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 960/1919] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:49.181826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.193590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd104ae-0c35-4885-8849-98ed74cf1ae0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.202229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c10403b098a475284d069f50d345043', 'request_id': 'req-f6cf219d-44bf-473a-a372-1d99e2e20a37'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:49.211948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c10403b098a475284d069f50d345043', 'request_id': 'req-f6cf219d-44bf-473a-a372-1d99e2e20a37'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:49.218864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c10403b098a475284d069f50d345043', 'request_id': 'req-f6cf219d-44bf-473a-a372-1d99e2e20a37'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:49.219975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:49.234750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] The token cache is being invalidate because project 2c10403b098a475284d069f50d345043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:49.235133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:49.236212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f6cf219d-44bf-473a-a372-1d99e2e20a37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:49.238323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 960/1920] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:49 2022] DELETE /identity/v3/projects/2c10403b098a475284d069f50d345043 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:49.242741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aa63-4084-4a66-b849-4b399fe4d884 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.243036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aa63-4084-4a66-b849-4b399fe4d884 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.243387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172aa63-4084-4a66-b849-4b399fe4d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.267390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a172aa63-4084-4a66-b849-4b399fe4d884 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:49.269491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a172aa63-4084-4a66-b849-4b399fe4d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.273895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.274758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.275099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.275671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] PATH_INFO: `/v3/projects/7dc27dde0b274004967ecb7ba3165a45/users/b907b4e8e57342dfaec1a507ba9b9983/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.278655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 961/1921] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:49.286157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.286444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.286725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.291925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc27dde0b274004967ecb7ba3165a45, user_id=b907b4e8e57342dfaec1a507ba9b9983, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.294613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.318563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 961/1922] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:49 2022] PUT /identity/v3/projects/7dc27dde0b274004967ecb7ba3165a45/users/b907b4e8e57342dfaec1a507ba9b9983/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:49.322593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.323832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:49.325264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.326224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.402119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a4869-e5de-4040-a9bd-41940e3e36f0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.406919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.407601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.407947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.408283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.409588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.411369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.412443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 962/1923] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:49.414816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.424353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 962/1924] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:49.426813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.428065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a9311c-b5a9-4dd4-95a0-7b12eefa2b92 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.434822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.435840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.436126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.436441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.437079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:49.437377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.437958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1895b8c4-827b-494b-bec7-d81e7a642a12 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.438963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.444193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.444912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.445184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.445460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] PATH_INFO: `/v3/projects/7dc27dde0b274004967ecb7ba3165a45/users/b907b4e8e57342dfaec1a507ba9b9983/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.449612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dc27dde0b274004967ecb7ba3165a45, user_id=b907b4e8e57342dfaec1a507ba9b9983, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.453015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fad60-b8d4-447f-9fd4-6d7545490904 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.460084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 963/1925] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:29:49.463062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.468950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 963/1926] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:49 2022] PUT /identity/v3/projects/7dc27dde0b274004967ecb7ba3165a45/users/b907b4e8e57342dfaec1a507ba9b9983/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:49.477923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.478200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.478470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.478768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eec0b4-f3e7-471e-aef0-687d6f767d61 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.511081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] MFA Rules not processed for user `b907b4e8e57342dfaec1a507ba9b9983`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:49.512189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.524473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:49.542314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.543232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.543796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.544076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.544770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.546660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.575277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 964/1927] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:49.580085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.584799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 964/1928] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:49.602637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.604521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.620899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:49.662546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d852555-09ee-4086-b6a0-7a7f9356b0a8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.668465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.669084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.669359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.669644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.670297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.672205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.677268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b4c49-cc59-45b0-8f83-9b91e94e2954 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.702087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 965/1929] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:49.708967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.720823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd12815-086b-4f30-9ae3-3c163d6989dd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.725873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.726600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.727242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.727609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.728390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.730305 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.743746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 966/1930] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:49.743920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.743920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.743920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.743920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.745119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.755432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed806d-f3e6-4f9a-ae46-ba9fadaeea25 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.760707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.761487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.761583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.761840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.762478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.764068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.776238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 967/1931] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:49.780536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.793042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cf6b2-91f8-4d5e-844e-19711fc99eca admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.800454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.801192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.801562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.801839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] PATH_INFO: `/v3/projects/a2fdd81304a7458cac5050ca53cabc4e/users/3a3650b9d95a48d18d1f303315640bbb/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.819715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2fdd81304a7458cac5050ca53cabc4e, user_id=3a3650b9d95a48d18d1f303315640bbb, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.822522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb25860-4d18-4a98-8ca0-0e303e008b59 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.827374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.829304 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.838964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 968/1932] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:49 2022] PUT /identity/v3/projects/a2fdd81304a7458cac5050ca53cabc4e/users/3a3650b9d95a48d18d1f303315640bbb/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:49.844944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.845230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.845499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.846142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 965/1933] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:49.884922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:49.884922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.894253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.905332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30d69d-2b99-44e0-b355-fcb59e9ea21d service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:49.964518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 969/1934] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:49.966857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:49.967580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:49.967862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:49.968141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] PATH_INFO: `/v3/users/f3f6a0aaf4e44abbaace037584d70972` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:49.973294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f6a0aaf4e44abbaace037584d70972)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:49.975282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:49.978934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:49.980367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:49.997592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:50.001484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:50.033255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f6a0aaf4e44abbaace037584d70972'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.042506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f6a0aaf4e44abbaace037584d70972'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.043279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f6a0aaf4e44abbaace037584d70972'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.054042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f6a0aaf4e44abbaace037584d70972'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.057857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0acea3-687d-4cf5-836f-f4edb322374d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:50.061853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:50.062471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.062750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.063019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:50.064821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:50.068437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c1453cfd-33c9-4028-8655-1a4cb8ef380e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f6a0aaf4e44abbaace037584d70972'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.084097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 966/1935] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:49 2022] DELETE /identity/v3/users/f3f6a0aaf4e44abbaace037584d70972 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:50.086173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:50.086450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.086763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.087146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:50.088067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:50.119227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:50.120763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:50.122798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 970/1936] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:50.193049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 967/1937] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:50.204440 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:50.205917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:50.217422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2426e-d405-4da6-b1da-bd8ac5fc0e32 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:50.272518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:50.273044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.273317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.273583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:50.283046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:50.285066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:50.294676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 971/1938] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:50.628748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:50.630640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:50.645155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd9430-37fb-4d8c-916b-17c47c417223 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:50.652357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:50.653194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.653568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.653947 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] PATH_INFO: `/v3/projects/6992f30c6d4c46f78ad0b0a15418809e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:50.655282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6992f30c6d4c46f78ad0b0a15418809e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:50.657350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:50.730417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6992f30c6d4c46f78ad0b0a15418809e', 'request_id': 'req-9a91f069-8628-48fb-951c-2c915d5fbe32'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.737868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6992f30c6d4c46f78ad0b0a15418809e', 'request_id': 'req-9a91f069-8628-48fb-951c-2c915d5fbe32'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.744043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6992f30c6d4c46f78ad0b0a15418809e', 'request_id': 'req-9a91f069-8628-48fb-951c-2c915d5fbe32'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.758476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] The token cache is being invalidate because project 6992f30c6d4c46f78ad0b0a15418809e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:50.758769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.759430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9a91f069-8628-48fb-951c-2c915d5fbe32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.760971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 968/1939] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:50 2022] DELETE /identity/v3/projects/6992f30c6d4c46f78ad0b0a15418809e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:50.765014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:50.775148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26445506-f22d-45f9-9d9b-4bce4d44137d service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:50.831830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:50.832519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.832775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.833120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] PATH_INFO: `/v3/users/b907b4e8e57342dfaec1a507ba9b9983` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:50.834403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b907b4e8e57342dfaec1a507ba9b9983)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:50.836388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:50.900680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b907b4e8e57342dfaec1a507ba9b9983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.927242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b907b4e8e57342dfaec1a507ba9b9983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.939725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:50.956620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b907b4e8e57342dfaec1a507ba9b9983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.957551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b907b4e8e57342dfaec1a507ba9b9983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.974123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b907b4e8e57342dfaec1a507ba9b9983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:50.986129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 972/1940] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:29:50 2022] DELETE /identity/v3/users/b907b4e8e57342dfaec1a507ba9b9983 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:50.992813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:50.993307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:50.993716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.035089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] MFA Rules not processed for user `3a3650b9d95a48d18d1f303315640bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:51.036716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.054828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:51.157534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 969/1941] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:51.161705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.163891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.300811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c335411-614a-4206-9936-d99005d63b59 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.306624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.307989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.308356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.308638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.309351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.311341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.369490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 973/1942] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:51 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:51.370559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.387479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3925e407-a398-4614-bd45-b56fc3711a7b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.493922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.498029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.573892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.574546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.575332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.576142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.576964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.579024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.605970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe6fba-a811-4372-9e2b-c59c8503d1bf admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.606207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 970/1943] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:51 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:51.609986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.611585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.612252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.612563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.613543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] PATH_INFO: `/v3/projects/7dc27dde0b274004967ecb7ba3165a45` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.614400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dc27dde0b274004967ecb7ba3165a45)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.616339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.631859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5bb43-9b29-4fa7-8bcb-9a8eb9bcf757 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.638312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.639035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.639955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.640866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.641659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.645298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.659530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 971/1944] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:51.663650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.671384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dc27dde0b274004967ecb7ba3165a45', 'request_id': 'req-ba704781-77ec-43a0-abf5-6ec99d45ab54'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:51.687165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b2cc1-ee08-41d2-b632-715ae76272f4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.693068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dc27dde0b274004967ecb7ba3165a45', 'request_id': 'req-ba704781-77ec-43a0-abf5-6ec99d45ab54'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:51.704975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dc27dde0b274004967ecb7ba3165a45', 'request_id': 'req-ba704781-77ec-43a0-abf5-6ec99d45ab54'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:51.707631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:51.720020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] The token cache is being invalidate because project 7dc27dde0b274004967ecb7ba3165a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:29:51.720323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:51.721093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ba704781-77ec-43a0-abf5-6ec99d45ab54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:29:51.723040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 974/1945] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:29:51 2022] DELETE /identity/v3/projects/7dc27dde0b274004967ecb7ba3165a45 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:51.727942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3100-8a60-4670-925c-43575fa4ad66 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.728223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3100-8a60-4670-925c-43575fa4ad66 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.728527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7c3100-8a60-4670-925c-43575fa4ad66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.748728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9d7c3100-8a60-4670-925c-43575fa4ad66 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:51.750578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9d7c3100-8a60-4670-925c-43575fa4ad66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.765400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.766067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.766345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.766618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] PATH_INFO: `/v3/projects/3be211315de849bead860d49c2ed0a63/users/76143a12ffc646f39d73050791f488d2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.781738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be211315de849bead860d49c2ed0a63, user_id=76143a12ffc646f39d73050791f488d2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.784461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.796813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 972/1946] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:51 2022] PUT /identity/v3/projects/3be211315de849bead860d49c2ed0a63/users/76143a12ffc646f39d73050791f488d2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:51.801244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.802861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.816122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:51.818586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 975/1947] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:51.827539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.827826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.828150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.847922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:51.849265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:51.873416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94328d-c820-48bc-a813-450bf80eaea9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.879403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.880033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.880331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.880606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.881260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.882966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.897625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 973/1948] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:51.898860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.908355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99846ac-d391-435d-90db-a2cef51a6f8f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.913757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.914392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.914668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.915055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] PATH_INFO: `/v3/projects/3be211315de849bead860d49c2ed0a63/users/76143a12ffc646f39d73050791f488d2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.916934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be211315de849bead860d49c2ed0a63, user_id=76143a12ffc646f39d73050791f488d2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.918403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 976/1949] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:51.919724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:51.922382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:51.931420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 974/1950] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:51 2022] PUT /identity/v3/projects/3be211315de849bead860d49c2ed0a63/users/76143a12ffc646f39d73050791f488d2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:29:51.935307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c5625-e5c1-4a52-88ae-f81a944e1527 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:51.993421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:51.994034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:51.994310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:51.994580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:51.995215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:29:51.995514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:51.997201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.009293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 977/1951] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:29:52.013819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.034120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c8de0-4a2c-4b2f-8f4d-defbd2102d02 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.039662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.040429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.040703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.041021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.041637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.043931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.075798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 975/1952] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:29:52.079928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.092634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac0f8fe-4b6e-48f1-b8ce-0bc7c30c24b9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.132270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:52.167890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.168574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.168855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.169126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.169782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.171745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.198286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 978/1953] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/users => generated 305 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:52.199738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.210424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6110c-b3ba-4f99-81cc-be2bfc8c417d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.215598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.216276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.216740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.217069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.217766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.219409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.233174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 976/1954] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:52.241449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.266364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f46199-3959-45bf-b1ea-d94295555ce8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.276954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.277768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.278087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.278324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.279343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.281602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9078abf9-96b7-44ae-8ef0-b0df2dd955b7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.298580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 979/1955] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:52.299600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.310937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50a5e7-4b70-4dc3-9168-bba58644d874 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.316357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.317100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.317445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.317806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] PATH_INFO: `/v3/projects/2c7d8fc1168546ea8cfd61bf8cd69a1b/users/6ec5e77762914f17a9a7a3c9edb44554/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.335983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7d8fc1168546ea8cfd61bf8cd69a1b, user_id=6ec5e77762914f17a9a7a3c9edb44554, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.339246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.363078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 977/1956] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:52 2022] PUT /identity/v3/projects/2c7d8fc1168546ea8cfd61bf8cd69a1b/users/6ec5e77762914f17a9a7a3c9edb44554/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:52.375049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15845b-3257-4f5e-9022-520913965745 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.375312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15845b-3257-4f5e-9022-520913965745 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.375587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15845b-3257-4f5e-9022-520913965745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.419039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2f15845b-3257-4f5e-9022-520913965745 None None] MFA Rules not processed for user `6ec5e77762914f17a9a7a3c9edb44554`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:52.420897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f15845b-3257-4f5e-9022-520913965745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:52.506640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 980/1957] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:52.507771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.642402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6564440-2f24-40c2-a843-ce94600a8346 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.648896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.649610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.650406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.650754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.651514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.654331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.691251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 978/1958] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/projects => generated 409 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:52.698102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.698458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.698801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.700358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.767964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] MFA Rules not processed for user `76143a12ffc646f39d73050791f488d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:52.770411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:52.770949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:52.829743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85017f88-7a3d-45a6-bd8f-b6038f1d0752 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.838211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.839120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.839773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.840252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.841124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.843640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.889144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 979/1959] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/users => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:29:52.892439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 981/1960] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:29:52.895411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.903153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.904660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:52.912023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db886fac-fa2a-44f8-8e39-3b1f5d264dfd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.924899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.925730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.926153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.926567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.927353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.929710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:52.947541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 980/1961] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:29:52.947737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:52.948868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:52.961329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f09bc5-5f58-4479-b5c8-2c7b890be5fd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.967850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.968570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.968837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.969255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] PATH_INFO: `/v3/projects/44dece259fd4420a9266f68a3a25af51/users/76512ca020cf43c7b9ecb685249cbee4/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.986634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91b9d3-6095-4381-88fc-8a940bf6b533 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:52.991910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:52.992655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:52.993128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:52.993407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:52.994180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44dece259fd4420a9266f68a3a25af51, user_id=76512ca020cf43c7b9ecb685249cbee4, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:52.995263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:52.997903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.032417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 981/1962] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:52 2022] PUT /identity/v3/projects/44dece259fd4420a9266f68a3a25af51/users/76512ca020cf43c7b9ecb685249cbee4/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:53.034166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:53.075388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:53.076853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.087041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 982/1963] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:53.106259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df0ff3-5176-47c0-908d-764dbd6aba1d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:53.110946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:53.111458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:53.111743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:53.112030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:53.112693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:53.114295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fa503-387e-4f22-98c7-784dad173c95 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.128762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 982/1964] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:29:53.129845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:53.141301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45094f3-774d-417b-a48c-5e88ec5c9c15 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:53.146781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:53.147538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:53.147909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:53.148248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] PATH_INFO: `/v3/projects/44dece259fd4420a9266f68a3a25af51/users/76512ca020cf43c7b9ecb685249cbee4/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:53.150168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44dece259fd4420a9266f68a3a25af51, user_id=76512ca020cf43c7b9ecb685249cbee4, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:53.152859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.164033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 983/1965] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:29:53 2022] PUT /identity/v3/projects/44dece259fd4420a9266f68a3a25af51/users/76512ca020cf43c7b9ecb685249cbee4/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:29:53.174250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:53.174523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:53.174928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:53.206805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] MFA Rules not processed for user `76512ca020cf43c7b9ecb685249cbee4`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:29:53.208023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:53.277517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:29:53.284124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 983/1966] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:29:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:29:53.294705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:53.296714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:29:53.388213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9e35a-df75-42f7-8221-ebf75ced93c4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:53.392250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:53.392902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:53.393178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:53.393474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:53.403554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:53.405036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.414608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 984/1967] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:53.575166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:53.586228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d018636-c14c-48ea-aaa9-ac9929335db6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:53.590903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:53.591583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:53.591926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:53.592342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:53.654162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:53.655612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:53.665734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 984/1968] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:29:55.538258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:29:55.547948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286a1266-0910-4924-af30-ac72f722e442 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:29:55.552191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:29:55.552817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:29:55.553087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:29:55.553383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:29:55.617429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:29:55.618920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1420efd-ca76-42bf-8d80-6cf254f42fb4 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:29:55.629065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 985/1969] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:02.267912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:02.404060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839266c-5af1-4832-aa41-36db862a9d8f service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:02.413352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:02.414325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:02.414742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:02.415162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] PATH_INFO: `/v3/users/6ec5e77762914f17a9a7a3c9edb44554` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:02.416546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec5e77762914f17a9a7a3c9edb44554)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:02.418941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:02.500407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec5e77762914f17a9a7a3c9edb44554'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:02.509438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec5e77762914f17a9a7a3c9edb44554'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:02.510219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec5e77762914f17a9a7a3c9edb44554'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:02.517955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec5e77762914f17a9a7a3c9edb44554'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:02.535195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec5e77762914f17a9a7a3c9edb44554'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:02.548560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 985/1970] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:02 2022] DELETE /identity/v3/users/6ec5e77762914f17a9a7a3c9edb44554 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:02.555244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:02.555803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:02.556436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:02.565221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:02.592987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:02.594382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:02.719650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 986/1971] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:02.767021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:02.774646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:02.834318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:02.924066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a73114-fd6a-4b30-93e1-2f9ddca2b451 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:02.934240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:02.934956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:02.935239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:02.935555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:02.951986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:02.954097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:02.968357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 986/1972] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:03.497778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:03.499762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:03.532763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8521cdd-ebda-4691-a17b-365adf5ee1df None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:03.552175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:03.552892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:03.553179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:03.554144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] PATH_INFO: `/v3/projects/2c7d8fc1168546ea8cfd61bf8cd69a1b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:03.555471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7d8fc1168546ea8cfd61bf8cd69a1b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:03.557608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:03.634510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7d8fc1168546ea8cfd61bf8cd69a1b', 'request_id': 'req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.649111 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7d8fc1168546ea8cfd61bf8cd69a1b', 'request_id': 'req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.657084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7d8fc1168546ea8cfd61bf8cd69a1b', 'request_id': 'req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.681665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] The token cache is being invalidate because project 2c7d8fc1168546ea8cfd61bf8cd69a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:03.682091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.683105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.686138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 987/1973] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:03 2022] DELETE /identity/v3/projects/2c7d8fc1168546ea8cfd61bf8cd69a1b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:03.689325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:03.701913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493a9e4-596a-48dd-b068-8ad9f4c8dddb service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:03.766748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:03.767389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:03.767660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:03.767932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] PATH_INFO: `/v3/users/76512ca020cf43c7b9ecb685249cbee4` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:03.769822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76512ca020cf43c7b9ecb685249cbee4)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:03.771822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:03.821331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76512ca020cf43c7b9ecb685249cbee4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.828823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76512ca020cf43c7b9ecb685249cbee4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.829526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76512ca020cf43c7b9ecb685249cbee4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.838324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76512ca020cf43c7b9ecb685249cbee4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.856432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ad7f230c-bd19-4ab7-bc3b-c7299960abc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76512ca020cf43c7b9ecb685249cbee4'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:03.869500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 987/1974] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:03 2022] DELETE /identity/v3/users/76512ca020cf43c7b9ecb685249cbee4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:04.299433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:04.301045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:04.316217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e9e50-fe91-4a4e-995c-fdb9fcecb0e3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:04.428722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:04.429998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:04.429998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:04.429998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] PATH_INFO: `/v3/projects/44dece259fd4420a9266f68a3a25af51` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:04.433349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=44dece259fd4420a9266f68a3a25af51)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:04.435274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:04.496199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:04.536873 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44dece259fd4420a9266f68a3a25af51', 'request_id': 'req-832eab54-1724-4595-8605-3094f5d075ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:04.563455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44dece259fd4420a9266f68a3a25af51', 'request_id': 'req-832eab54-1724-4595-8605-3094f5d075ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:04.592351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44dece259fd4420a9266f68a3a25af51', 'request_id': 'req-832eab54-1724-4595-8605-3094f5d075ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:04.653109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] The token cache is being invalidate because project 44dece259fd4420a9266f68a3a25af51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:04.653282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:04.661330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-832eab54-1724-4595-8605-3094f5d075ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:04.663952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 988/1975] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:04 2022] DELETE /identity/v3/projects/44dece259fd4420a9266f68a3a25af51 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:04.669301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:04.669599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:04.669875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:04.722954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:04.724398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:04.818666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:04.896930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 988/1976] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:04.905721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:04.906119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:04.906669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:04.984460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:04.992697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:05.032380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 989/1977] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:05.033994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.049142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe29388-1a42-4a88-8243-19ca7659c455 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.069474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.070103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.070543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.070974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.071778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:05.072226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.074095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.107755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 989/1978] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:05.112070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.130054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9ba5c-8089-4ca5-9c76-6cbfed1fe325 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.136575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.137316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.137805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.138140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.138794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.141116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.187705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 990/1979] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:05 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:05.189728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.203036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c0c978-0ccc-4208-ab8e-73db3277cf5e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.304906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.305526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.305808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.306083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.306937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.308944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.367791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 990/1980] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:05 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:05.373752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.391213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d207d3e6-9115-4a01-b173-fb62f1cf1fb0 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.406128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.406689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.406995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.407362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.411044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.416414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.441084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.448388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 991/1981] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:05.457531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3420c-a293-4940-8467-f548194b3db4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.472073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.472752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.473021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.473332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.473994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.475611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbcb4f-4c35-4605-abee-6a73ac447a9b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.491141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 991/1982] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:30:05.492753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.508834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38d7f6-5275-41c9-b945-bd5f424e5d61 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.515167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.515896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.516167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.516467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] PATH_INFO: `/v3/projects/6f05076be7044d2db90b9bf0cf45f8c6/users/162a364ee4f04928945755d59344633d/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.532464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f05076be7044d2db90b9bf0cf45f8c6, user_id=162a364ee4f04928945755d59344633d, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.535202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.573381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 992/1983] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:05 2022] PUT /identity/v3/projects/6f05076be7044d2db90b9bf0cf45f8c6/users/162a364ee4f04928945755d59344633d/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:05.578407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.578697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.578974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.634858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] MFA Rules not processed for user `162a364ee4f04928945755d59344633d`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:05.636522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:05.710665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 992/1984] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:05.721221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:05.723415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:05.794421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf7357-89bb-4866-8176-8b91b089b747 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:05.802291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:05.803010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:05.803278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:05.803543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:05.816536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:05.817980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8c263-a820-45e1-bfc2-01e077db0a44 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:05.827572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 993/1985] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:14.473692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:14.475098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:14.510182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:14.547333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824692ba-7fca-47ed-a933-f04a26495381 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:14.552113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:14.552770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:14.553048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:14.553318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] PATH_INFO: `/v3/users/1f45d4681cac4045a2a856044e636ecc` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:14.554438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f45d4681cac4045a2a856044e636ecc)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:14.556404 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:14.611887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f45d4681cac4045a2a856044e636ecc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:14.623764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f45d4681cac4045a2a856044e636ecc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:14.624633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f45d4681cac4045a2a856044e636ecc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:14.634695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f45d4681cac4045a2a856044e636ecc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:14.654724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f45d4681cac4045a2a856044e636ecc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:14.661091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 993/1986] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:14 2022] DELETE /identity/v3/users/1f45d4681cac4045a2a856044e636ecc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:14.665615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:14.666135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:14.666293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:14.685828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:14.687095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:14.743008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:14.765623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 994/1987] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:14.773728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:14.775169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:14.844365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b2cb7-1b69-471f-95fb-dd272521298f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:14.853781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:14.854817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:14.855101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:14.855379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:14.867305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:14.869343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e12ff9-c5e0-47cd-bb12-a0d341fec6fa service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:14.882869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 994/1988] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:15.298190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.301129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.369100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ef1a6-c3be-47e7-b6a8-d9d889c78cc8 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.380612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:15.382750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.383538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.383979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] PATH_INFO: `/v3/projects/2265e4f50bd34b37a0144856670df4cc` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.386698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2265e4f50bd34b37a0144856670df4cc)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:15.392081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:15.489242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2265e4f50bd34b37a0144856670df4cc', 'request_id': 'req-d0786a57-9261-48e8-9f73-528021c9c9cd'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.503513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2265e4f50bd34b37a0144856670df4cc', 'request_id': 'req-d0786a57-9261-48e8-9f73-528021c9c9cd'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.506813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2265e4f50bd34b37a0144856670df4cc', 'request_id': 'req-d0786a57-9261-48e8-9f73-528021c9c9cd'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.520021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] The token cache is being invalidate because project 2265e4f50bd34b37a0144856670df4cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:15.520420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.521130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-d0786a57-9261-48e8-9f73-528021c9c9cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.522805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 995/1989] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:15 2022] DELETE /identity/v3/projects/2265e4f50bd34b37a0144856670df4cc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:15.527393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.527854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.527939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.548193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:15.549417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.611349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:15.632564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 995/1990] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:15.640661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.640661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.640661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.667944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:15.667944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.676463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 996/1991] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:15.679841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.693359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebd4718-636c-4abd-940a-0a920ec67f37 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.698063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:15.698663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.698937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.699209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.699845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:15.700143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:15.701754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:15.713855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 996/1992] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:15.721166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.732589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e8c8b-5a65-46c3-96f1-068668f7fc96 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.739586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:15.739586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.739586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.739586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.739978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:15.740863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.741928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:15.743344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.756795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245090-06f1-4cf4-9dca-dfec5b650756 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.762536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:15.763547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.763836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.764234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] PATH_INFO: `/v3/users/44ab3ba5be3b45948167d69dcc3161eb` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.765635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ab3ba5be3b45948167d69dcc3161eb)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:15.767756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:15.776411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 997/1993] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:15 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:15.777614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.829722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ab3ba5be3b45948167d69dcc3161eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.845699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ab3ba5be3b45948167d69dcc3161eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.849591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ab3ba5be3b45948167d69dcc3161eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.864314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ab3ba5be3b45948167d69dcc3161eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.866084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52ba93-0a03-4321-bb4f-d921585bfb00 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.881233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:15.882410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.883103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.883526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.884331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:15.886695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:15.887074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c2529939-e7d2-475f-8e5b-76eef69b5605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ab3ba5be3b45948167d69dcc3161eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:15.896341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 997/1994] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:15 2022] DELETE /identity/v3/users/44ab3ba5be3b45948167d69dcc3161eb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:15.899995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:15.900478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:15.900918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:15.919125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 998/1995] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:15 2022] POST /identity/v3/users => generated 283 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:15.924267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:15.927049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.927481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:15.928983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:15.944894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b0ba-7d30-456a-bc04-a000e2fb7c53 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:15.996025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:16.005753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 998/1996] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:16.015764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.017961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:16.022025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.022678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.022950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.023220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.023902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.025613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.046458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 999/1997] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:16.047670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.060993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef168ff-6cfa-4f77-acfa-ccd1abfbbfdd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:16.066403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.067409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.067409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.067581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.068520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.069705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.080468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1000/1998] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:16.085250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.097464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb139eb4-67d4-4995-a34b-92932c96a639 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:16.098710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad10c6-8efd-49e0-b684-25c15faa9e71 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:16.102828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.103568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.103720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.103907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.104100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.104282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.104592 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.104868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] PATH_INFO: `/v3/projects/02428e22c6704c5dbe2021df863b9cd5/users/1ff146e8a8ec4490a4917b033e481440/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.114586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.116184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2212e5-8082-4c66-84af-f7f19f16d764 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.120466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:16.121116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02428e22c6704c5dbe2021df863b9cd5, user_id=1ff146e8a8ec4490a4917b033e481440, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.124161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.131765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 999/1999] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:16.143074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1001/2000] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:16 2022] PUT /identity/v3/projects/02428e22c6704c5dbe2021df863b9cd5/users/1ff146e8a8ec4490a4917b033e481440/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:16.153648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.153932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.154204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.184783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] MFA Rules not processed for user `1ff146e8a8ec4490a4917b033e481440`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:16.186085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:16.272223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1000/2001] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:16.284838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.286722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:16.356541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eeea02-a1d6-4528-87ac-b21d65ae4ec7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:16.361307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.362020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.362374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.362753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.376789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.378367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.389428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1002/2002] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:16.484429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.486333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:16.678050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f9974-3845-4950-975d-0177eb8ecb8e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:16.689105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:16.690830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:16.692089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:16.692549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] PATH_INFO: `/v3/projects/cd67711e9def4c7f89a21fc1d464a055` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:16.697166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd67711e9def4c7f89a21fc1d464a055)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:16.700187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:16.878533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd67711e9def4c7f89a21fc1d464a055', 'request_id': 'req-cc8cc880-97f4-4a87-89e0-ecda32530d29'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:16.898722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd67711e9def4c7f89a21fc1d464a055', 'request_id': 'req-cc8cc880-97f4-4a87-89e0-ecda32530d29'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:16.916115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd67711e9def4c7f89a21fc1d464a055', 'request_id': 'req-cc8cc880-97f4-4a87-89e0-ecda32530d29'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:16.937798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] The token cache is being invalidate because project cd67711e9def4c7f89a21fc1d464a055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:16.938318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:16.939098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:16.941446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1001/2003] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:16 2022] DELETE /identity/v3/projects/cd67711e9def4c7f89a21fc1d464a055 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:16.945732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:16.966462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf4c2d-f707-4f41-ac0b-6dc6ac6495ea service glance] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:17.157274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:17.161427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:17.161655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:17.162020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] PATH_INFO: `/v3/users/41ba50b350f949d98fd9615d38b1df83` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:17.163396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ba50b350f949d98fd9615d38b1df83)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:17.166262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:17.244476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:17.247417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ba50b350f949d98fd9615d38b1df83'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:17.272434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ba50b350f949d98fd9615d38b1df83'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:17.290754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ba50b350f949d98fd9615d38b1df83'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:17.293735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ba50b350f949d98fd9615d38b1df83'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:17.308925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe949bae-0127-49ba-bdb9-fe18d2781b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ba50b350f949d98fd9615d38b1df83'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:17.324196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1003/2004] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:16 2022] DELETE /identity/v3/users/41ba50b350f949d98fd9615d38b1df83 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:17.794683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:17.796076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:17.819927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8cc880-97f4-4a87-89e0-ecda32530d29 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:17.851438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:17.922399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:17.923290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:17.923737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:17.924485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] PATH_INFO: `/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:17.926030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f4da2fa32d64563b26e320e62a1ddbf)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:17.928213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:17.987207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f4da2fa32d64563b26e320e62a1ddbf', 'request_id': 'req-13f3b22d-667e-450a-bdff-2db7d350bb01'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.001087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f4da2fa32d64563b26e320e62a1ddbf', 'request_id': 'req-13f3b22d-667e-450a-bdff-2db7d350bb01'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.034889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f4da2fa32d64563b26e320e62a1ddbf', 'request_id': 'req-13f3b22d-667e-450a-bdff-2db7d350bb01'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.053576 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] The token cache is being invalidate because project 9f4da2fa32d64563b26e320e62a1ddbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:18.054009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.054678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.060038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1002/2005] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:17 2022] DELETE /identity/v3/projects/9f4da2fa32d64563b26e320e62a1ddbf => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:18.061501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be003b-f287-45a1-89e4-d6c14c0ecb57 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.061970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be003b-f287-45a1-89e4-d6c14c0ecb57 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.062251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be003b-f287-45a1-89e4-d6c14c0ecb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.098449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.099397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e3be003b-f287-45a1-89e4-d6c14c0ecb57 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:18.101656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.107389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e3be003b-f287-45a1-89e4-d6c14c0ecb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.311222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1004/2006] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:18.318950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3b22d-667e-450a-bdff-2db7d350bb01 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.322938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.323442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.323900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.331539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.332644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.333055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.333490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] PATH_INFO: `/v3/users/162a364ee4f04928945755d59344633d` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.336155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] RBAC: Authorizing `identity:delete_user(user_id=162a364ee4f04928945755d59344633d)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.339732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.374801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:18.376034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.392414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1005/2007] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:18.396353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.416373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1b1b3-e242-499c-b0cb-2b559814e039 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.431852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.440382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.440382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.440382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.444277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:18.444840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.449561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.516241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1006/2008] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:30:18.518168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.535720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd9f6c-7115-4278-aba1-ce8a3c7640c1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.535946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '162a364ee4f04928945755d59344633d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.553215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.557036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.558865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.560106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:18.561118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.578348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.578348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.582078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '162a364ee4f04928945755d59344633d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.584883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '162a364ee4f04928945755d59344633d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.598817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '162a364ee4f04928945755d59344633d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.627250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1007/2009] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:18 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:18.628918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-07f4aeb2-4829-4208-9641-eb82bc31a07b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '162a364ee4f04928945755d59344633d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:18.631441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.635259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.654141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1003/2010] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:18 2022] DELETE /identity/v3/users/162a364ee4f04928945755d59344633d => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:18.656353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8095966-7309-47db-89e0-f61029273348 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.656619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8095966-7309-47db-89e0-f61029273348 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.656893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8095966-7309-47db-89e0-f61029273348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.687597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a8095966-7309-47db-89e0-f61029273348 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:18.689284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a8095966-7309-47db-89e0-f61029273348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.790181 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c272e5-83e8-4ce0-a2cd-0f397c475752 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.814420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1004/2011] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:18.823668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8095966-7309-47db-89e0-f61029273348 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.825092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a8095966-7309-47db-89e0-f61029273348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.841017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:18.885955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.886809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.887120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.887469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.888194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.890237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.913978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1008/2012] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:18 2022] POST /identity/v3/users => generated 290 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:18.918002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.930729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe5d88-31cf-4583-98d5-185e694ef29e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.938643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.939816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.940324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.940798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.941740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.943627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.952109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8095966-7309-47db-89e0-f61029273348 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.956528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1009/2013] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:18.958147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:18.961199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:18.964036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.964845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.965269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.965549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.977201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e9b7f-920f-4f59-b0eb-84d2a86a2f24 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:18.982435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:18.983066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.983173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:18.983348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:18.983619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] PATH_INFO: `/v3/users/3a3650b9d95a48d18d1f303315640bbb` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:18.984794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3650b9d95a48d18d1f303315640bbb)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:18.984911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:18.986832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.005620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1005/2014] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:30:19.006480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.016178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f538cfa7-7e79-48a3-ad3c-aa13f7e09d80 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.021790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.022491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.023183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.023183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.023678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.025938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.037831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1006/2015] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:19.042740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.054979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9474cf8-75e1-4bb7-8649-6fd36f4b3e02 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.061128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.061932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.062218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.062489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] PATH_INFO: `/v3/projects/dcda2cbeec0245f889983fcdfc0e2d88/users/f0227df73e5a4f239f70f06b3ff0e8a6/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.074396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3650b9d95a48d18d1f303315640bbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.085292 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcda2cbeec0245f889983fcdfc0e2d88, user_id=f0227df73e5a4f239f70f06b3ff0e8a6, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.089563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3650b9d95a48d18d1f303315640bbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.101701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156ce1ce-1d22-4c70-9c54-737b0d33f138 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.113242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3650b9d95a48d18d1f303315640bbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.114207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3650b9d95a48d18d1f303315640bbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.126914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3650b9d95a48d18d1f303315640bbb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.135320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1007/2016] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:19 2022] PUT /identity/v3/projects/dcda2cbeec0245f889983fcdfc0e2d88/users/f0227df73e5a4f239f70f06b3ff0e8a6/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:19.137972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.137972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.137972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.142642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1010/2017] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:18 2022] DELETE /identity/v3/users/3a3650b9d95a48d18d1f303315640bbb => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:19.168212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] MFA Rules not processed for user `f0227df73e5a4f239f70f06b3ff0e8a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:19.169456 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.196776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:19.251522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1008/2018] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:19.257797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.259462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.355157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119576-a58e-4301-af5c-a3cea2e40acd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.360096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.360870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.361552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.361944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.363361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.365499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.406938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1011/2019] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:19 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:19.411803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.427373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2eeee-0b6c-4f5d-8136-6d36bcc13562 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.480585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.482428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.491918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:19.536625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.537437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.538264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.538682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.539409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.542203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.570550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1009/2020] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:19 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:19.574551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.586280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754f071-75aa-4459-8ccc-b27ae1925174 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.587012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe04d-0266-4a93-a2cd-0c690fed3526 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.591578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.592040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.592223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.592599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.592883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] PATH_INFO: `/v3/projects/a2fdd81304a7458cac5050ca53cabc4e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.593005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.593374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.593743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.594497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2fdd81304a7458cac5050ca53cabc4e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.594614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.596360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.596800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.618063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1010/2021] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:19.630675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.642547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae557bb-f40f-49a5-96a5-5d126825f027 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.650047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.650956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.651303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.651705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] PATH_INFO: `/v3/projects/0a5111a204e94d85af2b7e4e61570d71/users/7618cad76a1f459982c2368ea55ec074/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.690110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5111a204e94d85af2b7e4e61570d71, user_id=7618cad76a1f459982c2368ea55ec074, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.695930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.716523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2fdd81304a7458cac5050ca53cabc4e', 'request_id': 'req-0fe0b797-028c-4ed2-9f39-453ed96bb923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.728340 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2fdd81304a7458cac5050ca53cabc4e', 'request_id': 'req-0fe0b797-028c-4ed2-9f39-453ed96bb923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.734197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1011/2022] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:19 2022] PUT /identity/v3/projects/0a5111a204e94d85af2b7e4e61570d71/users/7618cad76a1f459982c2368ea55ec074/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:19.735023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.740001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.743832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2fdd81304a7458cac5050ca53cabc4e', 'request_id': 'req-0fe0b797-028c-4ed2-9f39-453ed96bb923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.762941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] The token cache is being invalidate because project a2fdd81304a7458cac5050ca53cabc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:19.763228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.763906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.768183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1012/2023] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:19 2022] DELETE /identity/v3/projects/a2fdd81304a7458cac5050ca53cabc4e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:19.768183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.768183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.841943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb74dc1-54ad-48a3-b522-46f6fbcf4943 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.848821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.849464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.849737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.850003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] PATH_INFO: `/v3/projects/6f05076be7044d2db90b9bf0cf45f8c6` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.851126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f05076be7044d2db90b9bf0cf45f8c6)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.853340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.871445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0b797-028c-4ed2-9f39-453ed96bb923 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.890581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:19.891335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:19.892046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:19.892046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:19.892799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:19.894583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:19.904978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1013/2024] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:19.909198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.911536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:19.912154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:19.939378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ec438-9791-4e4c-ae2b-900d9cf27255 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:19.951151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f05076be7044d2db90b9bf0cf45f8c6', 'request_id': 'req-f5cf92bd-1ade-4cac-a4dd-61e290952671'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.961283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f05076be7044d2db90b9bf0cf45f8c6', 'request_id': 'req-f5cf92bd-1ade-4cac-a4dd-61e290952671'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.969549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f05076be7044d2db90b9bf0cf45f8c6', 'request_id': 'req-f5cf92bd-1ade-4cac-a4dd-61e290952671'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.979834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:19.988707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] The token cache is being invalidate because project 6f05076be7044d2db90b9bf0cf45f8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:19.989070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.989815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:19.992005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1012/2025] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:19 2022] DELETE /identity/v3/projects/6f05076be7044d2db90b9bf0cf45f8c6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:19.993115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:19.997618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.020209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.021519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.021899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.022243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] PATH_INFO: `/v3/users/76143a12ffc646f39d73050791f488d2` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.023995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76143a12ffc646f39d73050791f488d2)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.026713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.071043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf92bd-1ade-4cac-a4dd-61e290952671 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.086816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.087818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.088266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.088718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] PATH_INFO: `/v3/projects/0a5111a204e94d85af2b7e4e61570d71/users/7618cad76a1f459982c2368ea55ec074/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.091186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76143a12ffc646f39d73050791f488d2'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:20.091644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a5111a204e94d85af2b7e4e61570d71, user_id=7618cad76a1f459982c2368ea55ec074, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.094038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01454065-45bc-48e2-a5d0-c1a19602d836 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.104185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76143a12ffc646f39d73050791f488d2'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:20.115741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1013/2026] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:19 2022] PUT /identity/v3/projects/0a5111a204e94d85af2b7e4e61570d71/users/7618cad76a1f459982c2368ea55ec074/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:20.117332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d69f-da11-4f41-b5b3-e70bea3aa699 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.117676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d69f-da11-4f41-b5b3-e70bea3aa699 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.117990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9d69f-da11-4f41-b5b3-e70bea3aa699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.122028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76143a12ffc646f39d73050791f488d2'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:20.122880 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76143a12ffc646f39d73050791f488d2'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:20.131304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a5550663-c194-4086-9226-b2977aec1892 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76143a12ffc646f39d73050791f488d2'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:20.138873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-aee9d69f-da11-4f41-b5b3-e70bea3aa699 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:20.139337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1014/2027] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:19 2022] DELETE /identity/v3/users/76143a12ffc646f39d73050791f488d2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:20.142318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-aee9d69f-da11-4f41-b5b3-e70bea3aa699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.142435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.142435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.142435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.185195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] MFA Rules not processed for user `7618cad76a1f459982c2368ea55ec074`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:20.187712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.280842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1014/2028] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:20.288980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.289402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.289775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.291095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1015/2029] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:20.303152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.304670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.318175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:20.320320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.332736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1015/2030] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:20.336929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.351362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429c0a-d193-42e2-a5fd-addbf2cdedfd None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.357910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.358775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.359132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.359474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.360329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:20.360862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.363075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.388481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:20.390227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1016/2031] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:30:20.395189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.408329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a2e9c-8f04-405a-86b5-befb05abd4b3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.416928 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.417670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.418019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.418362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.419070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.421344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.446256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce156e1-b1e6-411f-bd53-3f15a239dda9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.465284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.466116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.466493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.466864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.482677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.484438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.498333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1017/2032] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:20 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:20.502521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.509783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:20.511697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1016/2033] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:20.657991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8f498-9cda-40b5-b291-f7e60fb5fc22 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.661599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.673404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1845-f030-43fc-9950-da6e15076df3 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.684477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.685110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.685387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.685653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.687939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.692602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.786726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1018/2034] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:20 2022] POST /identity/v3/users => generated 288 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:20.790410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.790995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.792098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.793047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.793330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.799164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.824607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ee5f8-e2f7-4382-873b-b74967f7a16c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.840345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.841492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.841606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.841790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.842421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.844128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.889117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1019/2035] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:20.890129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.896688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:20.921811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f92a9a-8479-4757-ad1f-4500586af799 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.930051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.930536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.936316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.936316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] PATH_INFO: `/v3/projects/3be211315de849bead860d49c2ed0a63` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.937062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be211315de849bead860d49c2ed0a63)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.939435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.939802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.940936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:20.962399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1017/2036] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:20.963288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:20.978202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e74c7-95c7-46a4-a162-4cd499a77b69 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:20.984881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:20.985496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:20.985772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:20.986041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:20.986797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:20.990881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.021238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1018/2037] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:21.022125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.049126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f8968-7ed6-409f-8aef-78329446755f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.062696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.063379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.063658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.063936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] PATH_INFO: `/v3/projects/cb6fc9c33d0d4452892744fd265db15f/users/c6fcfe0cc7444f208fd6887e67cf5239/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.084651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6fc9c33d0d4452892744fd265db15f, user_id=c6fcfe0cc7444f208fd6887e67cf5239, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.087758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829962cb-dc21-4866-a5eb-1715d0dfbc9f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.100131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be211315de849bead860d49c2ed0a63', 'request_id': 'req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:21.120381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1019/2038] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:21 2022] PUT /identity/v3/projects/cb6fc9c33d0d4452892744fd265db15f/users/c6fcfe0cc7444f208fd6887e67cf5239/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:21.134979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be211315de849bead860d49c2ed0a63', 'request_id': 'req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:21.135556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.135834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.136125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.143854 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be211315de849bead860d49c2ed0a63', 'request_id': 'req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:21.188217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] The token cache is being invalidate because project 3be211315de849bead860d49c2ed0a63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:21.190879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:21.190879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9f7d18a9-1745-4f98-b960-b1c2cd4837e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:21.192527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1020/2039] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:20 2022] DELETE /identity/v3/projects/3be211315de849bead860d49c2ed0a63 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:21.204444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15ea2-fd66-435b-875c-bce97366b253 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.204729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15ea2-fd66-435b-875c-bce97366b253 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.205557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df15ea2-fd66-435b-875c-bce97366b253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.228650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] MFA Rules not processed for user `c6fcfe0cc7444f208fd6887e67cf5239`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:21.241377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:21.263892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0df15ea2-fd66-435b-875c-bce97366b253 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:21.270723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0df15ea2-fd66-435b-875c-bce97366b253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:21.354295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:21.399698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1020/2040] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:21.400827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.403587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:21.431377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1021/2041] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:21.457262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.457550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.457962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.487352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:21.519299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:21.520549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:21.528949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6799f3-bffe-4e8b-9d38-955c8c680a72 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.538928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.539816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.539978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1022/2042] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:21.540227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.540687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.541593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.543899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.545081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.556420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be508989-43bf-4a31-9393-062b4aa7cd47 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.562768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.563382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.563715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.563929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.564671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:21.564971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.566605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.578467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1021/2043] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:21.582466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.586111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1023/2044] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:21.587591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.597910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bd831-27d1-4486-9c88-fdcb1f13552d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.657034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca130345-bb6f-412a-b4da-946c8a8599e1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.662363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.662970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.663251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.663524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.664157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.666098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.666236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.666770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.667044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.667312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.668333 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.670098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.690656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1022/2045] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:21.694699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.699944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1024/2046] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:21.703842 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.704263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab70627-f3d8-4280-aaa7-03ed954f4fe2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.772033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643546-805f-447f-bacc-351709530142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.772867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643546-805f-447f-bacc-351709530142 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.773311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643546-805f-447f-bacc-351709530142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.773591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6643546-805f-447f-bacc-351709530142 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.774287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643546-805f-447f-bacc-351709530142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.776009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6643546-805f-447f-bacc-351709530142 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.779461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:21.781355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113f660-9756-4a38-b5df-a0edb26ce85d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.787463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.788072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.788397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.788657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.789313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.791243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.794559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1023/2047] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:30:21.795508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643546-805f-447f-bacc-351709530142 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.805956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6643546-805f-447f-bacc-351709530142 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.811437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.812090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.812398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.812670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] PATH_INFO: `/v3/projects/d6e66163ff024f309225ae583baa4237/users/a631af2b14e8435da1890c4b73e9d00c/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.815830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1025/2048] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:21 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:21.820043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.828373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e66163ff024f309225ae583baa4237, user_id=a631af2b14e8435da1890c4b73e9d00c, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.830393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9331fd-01bd-427b-82d0-7a459336dfc5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.833505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.836802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.837479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.837883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.838041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.838686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.840560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.852675 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1026/2049] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:30:21.856161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.865450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1024/2050] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:21 2022] PUT /identity/v3/projects/d6e66163ff024f309225ae583baa4237/users/a631af2b14e8435da1890c4b73e9d00c/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:21.866357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1e82-dc04-442c-8e02-b2b09bcf67b8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.869233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.924996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:21.940543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.941529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.941529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.941734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.942748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:21.944887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:21.966959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1027/2051] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:21.971030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:21.984165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77f2bd9-b226-49ae-befb-aca6543cb712 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.990133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa488e7e-ea34-4f0c-9bf1-144c0cef3e44 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:21.990277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.990952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.991227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.991499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] PATH_INFO: `/v3/projects/36e73ab908dd45449a62274bf18aac2d/users/b64401fba87941569a60f824a347059a/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:21.998068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:21.998815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:21.999172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:21.999548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.000531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.002819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.016902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e73ab908dd45449a62274bf18aac2d, user_id=b64401fba87941569a60f824a347059a, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.017436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1025/2052] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:22.021939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:22.023140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f9500-9740-4812-b848-e6b2ad2fa1dd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.033533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb085dc6-e9cc-4733-99cb-c56207ca0782 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:22.038690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1028/2053] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:21 2022] PUT /identity/v3/projects/36e73ab908dd45449a62274bf18aac2d/users/b64401fba87941569a60f824a347059a/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:22.050397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.050876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.051331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.083629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] MFA Rules not processed for user `b64401fba87941569a60f824a347059a`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:22.085132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.102770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:22.103568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.103924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.104361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] PATH_INFO: `/v3/projects/d6e66163ff024f309225ae583baa4237/users/a631af2b14e8435da1890c4b73e9d00c/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.107852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6e66163ff024f309225ae583baa4237, user_id=a631af2b14e8435da1890c4b73e9d00c, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.110527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb82ca6-9f25-4602-aed2-0718e04276a1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.121922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1026/2054] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:22 2022] PUT /identity/v3/projects/d6e66163ff024f309225ae583baa4237/users/a631af2b14e8435da1890c4b73e9d00c/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:22.132925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.133213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.133492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.162024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1029/2055] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:22.164859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] MFA Rules not processed for user `a631af2b14e8435da1890c4b73e9d00c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:22.166153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.170277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:22.172814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.224536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:22.237594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c00e7c-c6cd-417f-880a-7a9381bbd3df None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:22.244811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1027/2056] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:22.258056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:22.259759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.289945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:22.290566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.290847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.291131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.311915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:22.324306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223c4c19-cd03-46de-9327-0eb5aa958b02 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:22.328240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:22.328878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.329221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.329558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.357601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.359049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.368888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1030/2057] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:30:22.394141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:22.406665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.408319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6bc6e-cc35-4542-ba6d-fb58aa6d5e7b service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.424524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1028/2058] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:30:22.480917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45becb20-88f8-4a52-8e14-375d90f31b7e service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:22.486971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:22.488584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.489011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.489455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] PATH_INFO: `/v3/users/b64401fba87941569a60f824a347059a` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.490876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64401fba87941569a60f824a347059a)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.493435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.600214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64401fba87941569a60f824a347059a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:22.616831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64401fba87941569a60f824a347059a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:22.617584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64401fba87941569a60f824a347059a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:22.627224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64401fba87941569a60f824a347059a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:22.650815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64401fba87941569a60f824a347059a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:22.660885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1031/2059] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:22 2022] DELETE /identity/v3/users/b64401fba87941569a60f824a347059a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:22.665377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.665776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.666144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.696824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:22.698350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.783314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1029/2060] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:22.798213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:22.799959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:22.892775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:22.914890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d8956-56c8-420e-85a0-314c864fbf8a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:22.923870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:22.927980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:22.928330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:22.928894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:22.944627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:22.944627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed1ede5-cbbe-46d0-83d8-e16e2375bf26 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:22.956733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1032/2061] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:23.260065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.262411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:23.284175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c97ee8-1060-4654-bfc6-9fdc0b8270d5 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.289543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.290174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.290452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.290725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] PATH_INFO: `/v3/projects/36e73ab908dd45449a62274bf18aac2d` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.292470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e73ab908dd45449a62274bf18aac2d)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.294347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.349669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:23.360138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e73ab908dd45449a62274bf18aac2d', 'request_id': 'req-a3b72dba-862f-4926-a445-373c171c91ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:23.368944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e73ab908dd45449a62274bf18aac2d', 'request_id': 'req-a3b72dba-862f-4926-a445-373c171c91ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:23.376159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e73ab908dd45449a62274bf18aac2d', 'request_id': 'req-a3b72dba-862f-4926-a445-373c171c91ce'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:23.388505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] The token cache is being invalidate because project 36e73ab908dd45449a62274bf18aac2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:23.388798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:23.389517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a3b72dba-862f-4926-a445-373c171c91ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:23.391146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1030/2062] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:23 2022] DELETE /identity/v3/projects/36e73ab908dd45449a62274bf18aac2d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:23.397734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.398047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.398318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.423681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:23.424963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:23.483419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1033/2063] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:23.491809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.492145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.492400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.511945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:23.513219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:23.521757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1031/2064] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:23.526290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.536567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c096279-682c-4ff5-9b1f-f63dbab094a6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.541255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.541877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.542158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.542429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.543178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:23.543483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.545179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.557361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1034/2065] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:23.562221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.572323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad6f38-4da3-43f5-9998-6afaad512b60 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.577384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.577925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.578290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.578462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.579096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.581048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.613900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1032/2066] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:23 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:23.618243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.629369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002b110-f65c-4067-ba91-33fb402a8801 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.694210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.694823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.695102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.695375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.696003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.697952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.720790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1035/2067] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:23 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:23.724881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.735544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65518e-a7de-4a58-9a9f-1b00462db9a6 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.740155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.740779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.741056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.741326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.742076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.743659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.754176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1033/2068] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:30:23.758080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.768228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a29ec-421a-4d30-9980-ba5ea44accc0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.773939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.774608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.774914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.775193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.775896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.777615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a15f22-4a47-414c-9b09-d9dd85b4d73a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.788737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1036/2069] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:23.793008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.803116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87401229-6313-48d6-b5c0-4f144935a9b1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:23.807773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:23.808454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.808732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.809008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] PATH_INFO: `/v3/projects/0b4ce50640aa4e85ac515553c1922b8e/users/5e775bbd49024af5b9783515b315545a/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.824061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4ce50640aa4e85ac515553c1922b8e, user_id=5e775bbd49024af5b9783515b315545a, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:23.826724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:23.839901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1034/2070] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:23 2022] PUT /identity/v3/projects/0b4ce50640aa4e85ac515553c1922b8e/users/5e775bbd49024af5b9783515b315545a/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:23.851195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:23.851478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:23.851753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:23.884666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] MFA Rules not processed for user `5e775bbd49024af5b9783515b315545a`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:23.885873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:23.925664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:23.953228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1037/2071] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:23.964049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:23.965931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:24.033043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50caf2c-b5a3-456e-9bbe-f12bf61574a6 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:24.036885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:24.037481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:24.037759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:24.038024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:24.047101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:24.048547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee0b3-3574-4f0d-90ba-0a9be0689780 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:24.058467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1035/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:25.416695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:25.419698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:25.490802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abc59-e0b4-49ed-9d00-fd1986bdab63 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:25.495681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:25.496336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:25.496619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:25.496888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] PATH_INFO: `/v3/users/c6fcfe0cc7444f208fd6887e67cf5239` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:25.498010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6fcfe0cc7444f208fd6887e67cf5239)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:25.499953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:25.554452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6fcfe0cc7444f208fd6887e67cf5239'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:25.563436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6fcfe0cc7444f208fd6887e67cf5239'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:25.564044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6fcfe0cc7444f208fd6887e67cf5239'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:25.572307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6fcfe0cc7444f208fd6887e67cf5239'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:25.584286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6fcfe0cc7444f208fd6887e67cf5239'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:25.590167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1038/2073] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:25 2022] DELETE /identity/v3/users/c6fcfe0cc7444f208fd6887e67cf5239 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:25.594641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:25.595123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:25.595203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:25.616271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:25.617588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:25.682040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1036/2074] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:25.692519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:25.693987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:25.769337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088651c2-c256-484f-8133-8de5cee7cb10 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:25.773384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:25.774000 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:25.774272 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:25.774535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:25.784883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:25.786328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:25.792179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:25.798475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1039/2075] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:26.451931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:26.453362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:26.473023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432bbee-567f-4025-bb67-755713ed6074 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:26.482357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:26.487372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:26.487372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:26.487372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] PATH_INFO: `/v3/projects/cb6fc9c33d0d4452892744fd265db15f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:26.487372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6fc9c33d0d4452892744fd265db15f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:26.487372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:26.488481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:26.553610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6fc9c33d0d4452892744fd265db15f', 'request_id': 'req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.564142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6fc9c33d0d4452892744fd265db15f', 'request_id': 'req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.571519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6fc9c33d0d4452892744fd265db15f', 'request_id': 'req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.587569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] The token cache is being invalidate because project cb6fc9c33d0d4452892744fd265db15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:26.589272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.590543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.594187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1037/2076] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:26 2022] DELETE /identity/v3/projects/cb6fc9c33d0d4452892744fd265db15f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:26.598529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:26.614270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607190-34e6-49f5-bf81-e8e73d8fb011 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:26.721805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:26.722436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:26.722792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:26.723068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] PATH_INFO: `/v3/users/a631af2b14e8435da1890c4b73e9d00c` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:26.724430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a631af2b14e8435da1890c4b73e9d00c)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:26.726635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:26.780326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a631af2b14e8435da1890c4b73e9d00c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.789461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a631af2b14e8435da1890c4b73e9d00c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.790089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a631af2b14e8435da1890c4b73e9d00c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.798189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a631af2b14e8435da1890c4b73e9d00c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.812972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2063d928-9e52-408f-89f8-e4ad70e30505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a631af2b14e8435da1890c4b73e9d00c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:26.820329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1040/2077] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:26 2022] DELETE /identity/v3/users/a631af2b14e8435da1890c4b73e9d00c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:27.129170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.130853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:27.147031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052b4f7-3b30-43b4-99ce-4ebb6d8c1b33 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.214319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.215625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.215910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.216182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] PATH_INFO: `/v3/projects/d6e66163ff024f309225ae583baa4237` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.217358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6e66163ff024f309225ae583baa4237)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.219264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.291639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6e66163ff024f309225ae583baa4237', 'request_id': 'req-b99a8bd9-d716-4943-90c1-5475e56f733e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:27.304046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6e66163ff024f309225ae583baa4237', 'request_id': 'req-b99a8bd9-d716-4943-90c1-5475e56f733e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:27.310307 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6e66163ff024f309225ae583baa4237', 'request_id': 'req-b99a8bd9-d716-4943-90c1-5475e56f733e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:27.324734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] The token cache is being invalidate because project d6e66163ff024f309225ae583baa4237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:27.325121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:27.325653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b99a8bd9-d716-4943-90c1-5475e56f733e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:27.327366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1038/2078] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:27 2022] DELETE /identity/v3/projects/d6e66163ff024f309225ae583baa4237 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:27.332346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.332835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.333102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.361727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:27.363619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:27.426111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1041/2079] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:27.433766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.434052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.434328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.458606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:27.459969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:27.470295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1039/2080] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:27.475834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.489998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49395c2-4558-41dd-83bf-598f4b0d4826 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.509538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.510232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.510771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.511114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.511943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:27.512332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.514168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.535309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1042/2081] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:27.540696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.551746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38942c-7e04-4ba6-8646-9b365d8cf5ed None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.556751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.557348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.557624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.557881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.558498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.560526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.575755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:27.594218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1040/2082] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:27 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:27.611071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.626601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca1ed3-c73d-4313-bb4d-6ee224aa4ad5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.632320 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:27.796109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.796752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.797283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.797571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.798199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.800286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.840790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1043/2083] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:27 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:27.847036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.869455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edf8d6-8a65-4c00-bb8f-6d09b5ed4241 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.876312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.877034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.877387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.877829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.878625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.880801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.896359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1041/2084] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:27.902560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.919312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05aedaa-292c-4a9d-9cd2-0fe17460b316 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.926448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.927579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.927886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.928166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:27.928976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:27.931308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ef24b-6cb2-43f1-8a15-eca85ae589de admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:27.947919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1044/2085] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:27.951762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:27.963024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5192ef-5ce6-454b-b060-467c74dca9fb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:27.977884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:27.978577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:27.978852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:27.979123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] PATH_INFO: `/v3/projects/299f1a0143cc4e99a5402f020640bae0/users/5119931234d248e3aefeec7904598680/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:28.028643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=299f1a0143cc4e99a5402f020640bae0, user_id=5119931234d248e3aefeec7904598680, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:28.033474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:28.058918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1042/2086] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:27 2022] PUT /identity/v3/projects/299f1a0143cc4e99a5402f020640bae0/users/5119931234d248e3aefeec7904598680/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:28.071984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:28.072491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:28.073162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:28.115783 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] MFA Rules not processed for user `5119931234d248e3aefeec7904598680`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:28.117054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:28.259062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1045/2087] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:28.273761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:28.277463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:28.365278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3ac06-97b5-4daa-aec5-46a037f42e4d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:28.376359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:28.376992 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:28.377273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:28.377547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:28.387173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:28.388862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:28.399544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1043/2088] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:28.442733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:28.444524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:28.517937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc551b7-2e6c-425a-a635-2ee7d6e492ae None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:28.523983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:28.525155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:28.525155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:28.525955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] PATH_INFO: `/v3/users/e125a85011fe402ab2d5782d8fd39dfc` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:28.527121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e125a85011fe402ab2d5782d8fd39dfc)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:28.529370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:28.557429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:28.592391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e125a85011fe402ab2d5782d8fd39dfc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:28.601251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e125a85011fe402ab2d5782d8fd39dfc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:28.602643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e125a85011fe402ab2d5782d8fd39dfc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:28.618737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e125a85011fe402ab2d5782d8fd39dfc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:28.641036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-4c4f6be3-839e-492f-bd60-aff6b5a2f26b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e125a85011fe402ab2d5782d8fd39dfc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:28.656882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1046/2089] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:28 2022] DELETE /identity/v3/users/e125a85011fe402ab2d5782d8fd39dfc => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:28.997483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.015580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13124b6-be9a-4aaa-8b79-343b222b6fd4 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.115334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.116505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.116798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.117057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] PATH_INFO: `/v3/projects/acfd00e7112b409bba3cdf9480508b02` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.118172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfd00e7112b409bba3cdf9480508b02)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.120138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.173188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfd00e7112b409bba3cdf9480508b02', 'request_id': 'req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:29.184042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfd00e7112b409bba3cdf9480508b02', 'request_id': 'req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:29.192062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:29.195915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfd00e7112b409bba3cdf9480508b02', 'request_id': 'req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:29.214718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] The token cache is being invalidate because project acfd00e7112b409bba3cdf9480508b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:29.215109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:29.215897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4c2e51dd-6b38-4861-adb5-7dbccd4b6923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:29.233210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1044/2090] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:28 2022] DELETE /identity/v3/projects/acfd00e7112b409bba3cdf9480508b02 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:29.234941 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259741e-607a-4545-8973-700936eab5f9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.235275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259741e-607a-4545-8973-700936eab5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.235647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259741e-607a-4545-8973-700936eab5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.266247 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0259741e-607a-4545-8973-700936eab5f9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:29.277816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0259741e-607a-4545-8973-700936eab5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:29.369868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1047/2091] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:29.377860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.378354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.379451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.403435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:29.404645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:29.415206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1045/2092] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:29.419883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259741e-607a-4545-8973-700936eab5f9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.429872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259741e-607a-4545-8973-700936eab5f9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.434472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.435060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.435334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.435599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.436229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:29.436557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.438135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.450725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1048/2093] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:29.454969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.466821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098eca63-d51a-4add-9ed2-2c2996b1992f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.471437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.472105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.472387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.472658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.473287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.475179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.503067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1046/2094] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:29 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:29.507755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.518770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d5c15-70dd-4e4c-be0b-37ac25798052 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.576572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.577175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.577452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.577723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.578344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.580245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.601228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1049/2095] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:29 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:29.605916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.617141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa46ddc-a5e1-403e-893f-779bbd58af5d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.623422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.624044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.624463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.624733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.625366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.626947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.637354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1047/2096] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:29.641372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.654627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f9d80-233e-4522-9411-8a86c588e301 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.659380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.660750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.661225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.661703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.662825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.664872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de25cfd-3615-4aa5-9453-c18ce9cf6270 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.671232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:29.677973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1050/2097] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:29.681786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.691800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b779-7396-4b05-b0e0-3fc9d99f589f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.696450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.697085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.697364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.697633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] PATH_INFO: `/v3/projects/553395befb8f4c958d34b7dffb2317e7/users/262e5327ba144a6287fef6dd6293f886/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.714009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553395befb8f4c958d34b7dffb2317e7, user_id=262e5327ba144a6287fef6dd6293f886, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.716710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.730918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1048/2098] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:29 2022] PUT /identity/v3/projects/553395befb8f4c958d34b7dffb2317e7/users/262e5327ba144a6287fef6dd6293f886/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:29.736027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.736318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.736604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.755869 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:29.757075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:29.816992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1051/2099] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:29.826537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:29.828276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:29.890106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ff9ce8-ea62-4dc9-9da6-2cb54dee6151 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:29.894967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:29.895683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:29.895859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:29.896156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:29.905723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:29.907145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de177b2c-a588-4f2a-b9fd-4e676c4beb85 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:29.917110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1049/2100] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:34.242697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:34.244166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:34.271992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6b563-81dc-400d-925a-5290af729235 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:34.279731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:34.280537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:34.280826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:34.281200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] PATH_INFO: `/v3/users/f0227df73e5a4f239f70f06b3ff0e8a6` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:34.283361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0227df73e5a4f239f70f06b3ff0e8a6)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:34.297397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:34.406223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0227df73e5a4f239f70f06b3ff0e8a6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:34.419508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0227df73e5a4f239f70f06b3ff0e8a6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:34.422878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0227df73e5a4f239f70f06b3ff0e8a6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:34.453486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0227df73e5a4f239f70f06b3ff0e8a6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:34.507420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0227df73e5a4f239f70f06b3ff0e8a6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:34.520097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1052/2101] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:34 2022] DELETE /identity/v3/users/f0227df73e5a4f239f70f06b3ff0e8a6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:34.526048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f782fb-6b07-4d40-b945-4cdb4537db6d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:34.526048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f782fb-6b07-4d40-b945-4cdb4537db6d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:34.526048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f782fb-6b07-4d40-b945-4cdb4537db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:34.587644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c7f782fb-6b07-4d40-b945-4cdb4537db6d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:34.590538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c7f782fb-6b07-4d40-b945-4cdb4537db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:34.757069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:34.760202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:34.762450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1050/2102] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:35.017191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d1776-ac0a-46b4-8fb4-59fb9ca0ff44 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:35.044370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:35.044370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:35.044370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:35.044370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:35.070727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:35.073771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:35.096721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1053/2103] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:30:35.381958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:35.381958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:35.381958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:35.424045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] MFA Rules not processed for user `262e5327ba144a6287fef6dd6293f886`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:35.425886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:35.488986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:35.617013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1051/2104] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:35.627609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:35.631136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:35.636098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:35.639172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:35.663517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb51ee-0684-4b9b-93da-474b2a2244dc service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:35.681714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:35.682517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:35.682890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:35.683360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] PATH_INFO: `/v3/projects/dcda2cbeec0245f889983fcdfc0e2d88` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:35.689483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcda2cbeec0245f889983fcdfc0e2d88)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:35.693057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:35.714943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:35.778512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1386-8f90-414c-a1a9-38cdcdaea8bd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:35.797265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcda2cbeec0245f889983fcdfc0e2d88', 'request_id': 'req-f445f3d9-3747-4833-bcad-40b94ef260e9'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:35.814021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcda2cbeec0245f889983fcdfc0e2d88', 'request_id': 'req-f445f3d9-3747-4833-bcad-40b94ef260e9'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:35.828723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcda2cbeec0245f889983fcdfc0e2d88', 'request_id': 'req-f445f3d9-3747-4833-bcad-40b94ef260e9'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:35.867298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] The token cache is being invalidate because project dcda2cbeec0245f889983fcdfc0e2d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:35.868090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:35.871121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:35.877420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1054/2105] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:35 2022] DELETE /identity/v3/projects/dcda2cbeec0245f889983fcdfc0e2d88 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:35.887750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:35.890212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:35.931441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:35.932326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:35.932720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:35.933044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:35.934927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:36.082263 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445f3d9-3747-4833-bcad-40b94ef260e9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:36.084769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:36.094452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:36.096801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:36.125252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:36.125707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1052/2106] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:30:36.141328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:36.143297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:36.144270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] PATH_INFO: `/v3/users/7618cad76a1f459982c2368ea55ec074` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:36.151326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7618cad76a1f459982c2368ea55ec074)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:36.155533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:36.244003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7618cad76a1f459982c2368ea55ec074'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:36.258599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7618cad76a1f459982c2368ea55ec074'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:36.259511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7618cad76a1f459982c2368ea55ec074'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:36.284016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7618cad76a1f459982c2368ea55ec074'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:36.320698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-51b26419-de9f-4ca1-82b3-69dc7951235e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7618cad76a1f459982c2368ea55ec074'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:36.329959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1055/2107] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:35 2022] DELETE /identity/v3/users/7618cad76a1f459982c2368ea55ec074 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:36.892620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:36.921752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b1d4c-1ba1-4060-99a3-a649db8c1be7 service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:37.114722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:37.115657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.115937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.116212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] PATH_INFO: `/v3/projects/0a5111a204e94d85af2b7e4e61570d71` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.118834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a5111a204e94d85af2b7e4e61570d71)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:37.121456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:37.268642 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a5111a204e94d85af2b7e4e61570d71', 'request_id': 'req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:37.293757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a5111a204e94d85af2b7e4e61570d71', 'request_id': 'req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:37.317588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a5111a204e94d85af2b7e4e61570d71', 'request_id': 'req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:37.335722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] The token cache is being invalidate because project 0a5111a204e94d85af2b7e4e61570d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:37.336364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:37.337361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-820bc2a7-c1c1-469d-a622-e18f3d78cf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:37.340393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1053/2108] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:36 2022] DELETE /identity/v3/projects/0a5111a204e94d85af2b7e4e61570d71 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:37.345303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.348060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.349218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.388069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:37.389944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:37.423291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:37.548494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1056/2109] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:37.549476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.549767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.550155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.601663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:37.603407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:37.628493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1054/2110] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:30:37.636454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:37.646726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27afe18a-b26e-4c02-aaa7-bc634aa15a20 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:37.658136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:37.659045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.660097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.661992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.663068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:37.663545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:37.665372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:37.685463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1057/2111] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:30:37.690288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:37.699784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924e453-7c3e-467e-af27-3c1753ddf1e7 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:37.721380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:37.721380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.721380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.721876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.722735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:37.725851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:37.798847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:37.811515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1055/2112] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:37 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:37.817144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c040d7d9-858d-44d7-8273-e54185b15c12 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:37.978329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:37.979215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:37.979504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:37.979784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:37.980533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:37.982435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.053877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1058/2113] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:37 2022] POST /identity/v3/users => generated 283 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:30:38.058683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:38.091160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea8fbb-27d7-43ff-8cd6-ca1b3a664bfd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:38.118109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:38.123930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.124124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.125737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.127830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:38.133121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.163488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:38.174265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1056/2114] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:38.177151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5369c7-c62e-41c2-9e90-dcf7785c5221 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:38.189797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:38.191120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.191362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.191713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.192389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:38.198657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b3bb3-4a01-43cf-88a5-c635a6d199fd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.230991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:38.241237 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1059/2115] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:38.243673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67c6aa-f7e4-431c-a8cd-c39a4b3d2668 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:38.279923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:38.282659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.286449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.286731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] PATH_INFO: `/v3/projects/a25450f2eaee494d836c71ae0749fa47/users/08e75c1cb3f64effa401ca6ab1248e8e/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.353089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25450f2eaee494d836c71ae0749fa47, user_id=08e75c1cb3f64effa401ca6ab1248e8e, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:38.356113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.381252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1057/2116] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:38 2022] PUT /identity/v3/projects/a25450f2eaee494d836c71ae0749fa47/users/08e75c1cb3f64effa401ca6ab1248e8e/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:38.393070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8e37-4b6e-4770-979d-5deb3543206c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.393595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8e37-4b6e-4770-979d-5deb3543206c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.393727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c8e37-4b6e-4770-979d-5deb3543206c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.471058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-828c8e37-4b6e-4770-979d-5deb3543206c None None] MFA Rules not processed for user `08e75c1cb3f64effa401ca6ab1248e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:38.473649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-828c8e37-4b6e-4770-979d-5deb3543206c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:38.575863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1060/2117] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:38.580323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:38.615576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:38.687627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.687627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.687845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.732318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] MFA Rules not processed for user `262e5327ba144a6287fef6dd6293f886`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:38.732318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:38.742936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95cfcf6-7a5f-454c-99d5-90962117f2b4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:38.750785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:38.754160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:38.754794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.755528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.755651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.756225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:38.758724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.817041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1058/2118] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:38 2022] POST /identity/v3/projects => generated 365 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:38.820087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:38.880644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1061/2119] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:38.931374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c9d59-ebfa-4f0f-a693-f5434da73107 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:38.937509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:38.938313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:38.938829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:38.939106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:38.939729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:38.943980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:38.998841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1059/2120] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:38 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:39.003878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:39.022703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc647dc-3d16-4ab2-8d2b-8d5eccb75c32 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:39.034121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:39.036452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.036452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.036452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:39.036763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:39.046973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:39.069115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1062/2121] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:39.070302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:39.084317 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bbb4-c6b0-4339-9651-3fe1fcdb3627 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:39.094923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:39.095877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.096381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.097424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] PATH_INFO: `/v3/projects/9db194c5b57e499b83ccce3e75f5c262/users/f10f8bb0d91b4ad3951b6f82dba78e86/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:39.121331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db194c5b57e499b83ccce3e75f5c262, user_id=f10f8bb0d91b4ad3951b6f82dba78e86, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:39.125116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:39.152832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1060/2122] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:39 2022] PUT /identity/v3/projects/9db194c5b57e499b83ccce3e75f5c262/users/f10f8bb0d91b4ad3951b6f82dba78e86/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:39.157899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:39.176441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ac1b8-1880-40e8-a5f8-3ba6510ca86c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:39.453627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:39.454344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.454549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.455163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:39.456832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:39.458656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0bac08-d44a-4651-a826-2b77252108a3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:39.506331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1063/2123] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:39.508207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:39.526634 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3722b-e9b8-4860-9226-76acf728df6f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:39.537684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:39.539106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.539522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.539881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] PATH_INFO: `/v3/projects/9db194c5b57e499b83ccce3e75f5c262/users/f10f8bb0d91b4ad3951b6f82dba78e86/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:39.544091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db194c5b57e499b83ccce3e75f5c262, user_id=f10f8bb0d91b4ad3951b6f82dba78e86, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:39.550688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:39.587396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1061/2124] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:39 2022] PUT /identity/v3/projects/9db194c5b57e499b83ccce3e75f5c262/users/f10f8bb0d91b4ad3951b6f82dba78e86/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:39.597711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.598086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.598380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:39.650026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] MFA Rules not processed for user `f10f8bb0d91b4ad3951b6f82dba78e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:39.651584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:39.820357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1064/2125] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:39.821645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:39.825247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:39.931300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9050fdee-297b-4240-a9fa-2727e7b9465b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:39.958496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:39.959143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:39.961977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:39.961977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:40.019614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:40.034769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17633ca-79eb-44b6-a13e-b236f66e787c service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:40.069087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:40.073670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1062/2126] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:40.928100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:40.990185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:41.064608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b22ce-4a1b-48af-a641-33497a15215c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:41.085312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:41.085312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:41.085312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:41.085312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] PATH_INFO: `/v3/users/08e75c1cb3f64effa401ca6ab1248e8e` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:41.086183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e75c1cb3f64effa401ca6ab1248e8e)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:41.086183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:41.204792 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e75c1cb3f64effa401ca6ab1248e8e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:41.223322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e75c1cb3f64effa401ca6ab1248e8e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:41.224885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e75c1cb3f64effa401ca6ab1248e8e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:41.242416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e75c1cb3f64effa401ca6ab1248e8e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:41.277892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e75c1cb3f64effa401ca6ab1248e8e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:41.284972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1065/2127] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:40 2022] DELETE /identity/v3/users/08e75c1cb3f64effa401ca6ab1248e8e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:41.297094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:41.299654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:41.300589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:41.334698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:41.336266 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:41.429472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1063/2128] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:41.439987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:41.441632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:41.540262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed935e7-bc35-4034-b57c-b3d43ec3ebc7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:41.547373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:41.549163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:41.549651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:41.550369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:41.564044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:41.566583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:41.596257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1066/2129] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:41.984577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:41.987098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:42.004822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19265a8b-ebd5-4e15-ab69-3bda36d3e754 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:42.015882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:42.016804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:42.017084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:42.017354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] PATH_INFO: `/v3/projects/a25450f2eaee494d836c71ae0749fa47` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:42.018540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25450f2eaee494d836c71ae0749fa47)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:42.020733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:42.080965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25450f2eaee494d836c71ae0749fa47', 'request_id': 'req-94dbb82e-9b63-424f-a530-1990ac44ff14'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.091882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25450f2eaee494d836c71ae0749fa47', 'request_id': 'req-94dbb82e-9b63-424f-a530-1990ac44ff14'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.098355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25450f2eaee494d836c71ae0749fa47', 'request_id': 'req-94dbb82e-9b63-424f-a530-1990ac44ff14'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.110907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] The token cache is being invalidate because project a25450f2eaee494d836c71ae0749fa47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:42.111200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.111856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.113974 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1064/2130] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:41 2022] DELETE /identity/v3/projects/a25450f2eaee494d836c71ae0749fa47 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:42.118079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:42.128923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd158c-623c-489e-b411-8abc3218a2e3 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:42.179233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:42.193002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:42.193853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:42.194170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:42.194413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] PATH_INFO: `/v3/users/f10f8bb0d91b4ad3951b6f82dba78e86` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:42.195877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f10f8bb0d91b4ad3951b6f82dba78e86)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:42.198253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:42.259975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f10f8bb0d91b4ad3951b6f82dba78e86'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.269081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f10f8bb0d91b4ad3951b6f82dba78e86'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.269841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f10f8bb0d91b4ad3951b6f82dba78e86'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.278778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f10f8bb0d91b4ad3951b6f82dba78e86'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.297210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-67b09533-1062-4fad-acf8-286b30f5cb76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f10f8bb0d91b4ad3951b6f82dba78e86'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.309300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1067/2131] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:42 2022] DELETE /identity/v3/users/f10f8bb0d91b4ad3951b6f82dba78e86 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:42.332670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:42.334260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:42.350889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb82e-9b63-424f-a530-1990ac44ff14 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:42.469126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:42.469977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:42.471053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:42.472698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] PATH_INFO: `/v3/users/5e775bbd49024af5b9783515b315545a` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:42.474954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e775bbd49024af5b9783515b315545a)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:42.483329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:42.567684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:42.614944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e775bbd49024af5b9783515b315545a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.643954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e775bbd49024af5b9783515b315545a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.677919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e775bbd49024af5b9783515b315545a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.680007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e775bbd49024af5b9783515b315545a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.724934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e775bbd49024af5b9783515b315545a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:42.760409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1065/2132] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:30:42 2022] DELETE /identity/v3/users/5e775bbd49024af5b9783515b315545a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:42.765802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:42.766414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:42.766888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:42.802759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:42.806115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:42.919290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1068/2133] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:42.931421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:42.934094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:42.976014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:42.979134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:43.019074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a8f35-adbe-4559-b480-ab150cf870bd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.023387 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.024046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.024375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.024653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.038289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.039222 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2e368-ffe6-4a38-8f49-81270580bb40 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.040460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da27da-d8ea-4a66-88d2-47cb4eb9e0d6 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.054511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.055165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.055488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.055764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] PATH_INFO: `/v3/projects/9db194c5b57e499b83ccce3e75f5c262` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.056637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1066/2134] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:43.057054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db194c5b57e499b83ccce3e75f5c262)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.059013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.160182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db194c5b57e499b83ccce3e75f5c262', 'request_id': 'req-5c3c39c0-9749-49b9-ae16-837e2d61c123'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.174027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db194c5b57e499b83ccce3e75f5c262', 'request_id': 'req-5c3c39c0-9749-49b9-ae16-837e2d61c123'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.186748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db194c5b57e499b83ccce3e75f5c262', 'request_id': 'req-5c3c39c0-9749-49b9-ae16-837e2d61c123'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.189376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:43.210529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] The token cache is being invalidate because project 9db194c5b57e499b83ccce3e75f5c262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:43.211053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.211876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5c3c39c0-9749-49b9-ae16-837e2d61c123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.213849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1069/2135] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:42 2022] DELETE /identity/v3/projects/9db194c5b57e499b83ccce3e75f5c262 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:43.220579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.220946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.221223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.246420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:43.247778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:43.357459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1067/2136] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:43.366066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.366624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.367067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.401981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:43.407041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:43.423507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.425170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:43.433197 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1070/2137] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:43.437896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.438659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb19f7-aedb-42f9-a55d-f590837e2abe None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.444000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.444680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.444957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.445229 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] PATH_INFO: `/v3/projects/0b4ce50640aa4e85ac515553c1922b8e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.446549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4ce50640aa4e85ac515553c1922b8e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.449495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.460434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf41d7-1ef8-44fa-95af-6ac08de8d01f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.471817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.473256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.473598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.473932 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.476572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:30:43.478147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.479929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.510064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1071/2138] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:30:43.511433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.526165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74d352-7cd3-4b07-b096-7ede4336b3cc admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.528470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:43.542150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4ce50640aa4e85ac515553c1922b8e', 'request_id': 'req-a2bf6889-59e3-4d89-8a2d-8d225127f15e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.553248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4ce50640aa4e85ac515553c1922b8e', 'request_id': 'req-a2bf6889-59e3-4d89-8a2d-8d225127f15e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.561750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4ce50640aa4e85ac515553c1922b8e', 'request_id': 'req-a2bf6889-59e3-4d89-8a2d-8d225127f15e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.578820 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] The token cache is being invalidate because project 0b4ce50640aa4e85ac515553c1922b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:30:43.579247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.580656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:30:43.582442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1068/2139] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:30:43 2022] DELETE /identity/v3/projects/0b4ce50640aa4e85ac515553c1922b8e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:30:43.611686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.613028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.614115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.614115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.615123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.617937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.685039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1072/2140] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:43 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:43.689758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.693551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:43.718844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf6889-59e3-4d89-8a2d-8d225127f15e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.888369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.889289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.889640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.889917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.890589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.892672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.922201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1069/2141] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:43 2022] POST /identity/v3/users => generated 289 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:43.927132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.938134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a56c77-6a6c-4292-9d9c-f76adbb4480e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.943666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.944421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.944812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.945171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.945884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.947620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:43.969501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:43.976103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1073/2142] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:43.982140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bb555-3121-4598-b4ff-e51f77190b98 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:43.988574 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:43.989647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:43.989762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:43.991283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:43.991283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:43.993392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef6747-7a75-4564-b4c8-c01f69102dba admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.016809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1070/2143] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:44.021358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.033504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6dc17-c04c-4648-95c2-25e4538096a9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.045635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.045985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.046332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.046686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] PATH_INFO: `/v3/projects/df150e1afc044ef0b8479ccb7ca8dfc5/users/75a0e5b3262b4f399c1ea55ff4b354d9/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.064820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df150e1afc044ef0b8479ccb7ca8dfc5, user_id=75a0e5b3262b4f399c1ea55ff4b354d9, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.068601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.087085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1074/2144] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:44 2022] PUT /identity/v3/projects/df150e1afc044ef0b8479ccb7ca8dfc5/users/75a0e5b3262b4f399c1ea55ff4b354d9/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:44.098901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.099268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.099664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.130189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] MFA Rules not processed for user `75a0e5b3262b4f399c1ea55ff4b354d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:44.131739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:44.209423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1071/2145] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:44.210678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.280508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49b84-1601-4534-8ecf-072dcab69f45 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.286287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.287154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.287441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.287710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.288598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.290281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.324693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1075/2146] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:30:44 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:44.329161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.349334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cd9634-ea04-43fc-8378-e1a2d2f1feed None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.414092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:44.464622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.465290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.465571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.465844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.466503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.468448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.494095 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1072/2147] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:30:44 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:30:44.498357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.509088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a62361c-a2a5-4a9f-af13-2b7de642a9ad admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.513890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.514501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.514793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.515049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.515681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.518279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.521261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:30:44.533466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1076/2148] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:44.537955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.548654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690fdc-74be-4808-93fe-effad8cb1d3a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.553355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.554016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.554291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.554560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] PATH_INFO: `/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7/users/05139d3dc6494135b999f7d290639278/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.581913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2616ed3d543a4031a8ba3457dbcbe0f7, user_id=05139d3dc6494135b999f7d290639278, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.584901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.608349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1073/2149] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:44 2022] PUT /identity/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7/users/05139d3dc6494135b999f7d290639278/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:44.612130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.622434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac14d4f-bfe5-4104-a214-6867296745f2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.679698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.680315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.680598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.680859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.681482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.683051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719967b-c041-4e08-80c3-a023dcef179f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.693677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1077/2150] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:30:44.697697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.709488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa5bbc-35fc-4831-9181-d50d26210853 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.714101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.714744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.715156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.715433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] PATH_INFO: `/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7/users/05139d3dc6494135b999f7d290639278/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.717326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2616ed3d543a4031a8ba3457dbcbe0f7, user_id=05139d3dc6494135b999f7d290639278, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:44.720043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:44.732784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1074/2151] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:30:44 2022] PUT /identity/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7/users/05139d3dc6494135b999f7d290639278/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:30:44.743693 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.743983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.744264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:44.773484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] MFA Rules not processed for user `05139d3dc6494135b999f7d290639278`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:30:44.774676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:44.844447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1078/2152] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:30:44.853372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:44.856643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:44.932092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57366a8f-8c60-4ef4-81b1-df5e2458286b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:44.935799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:44.936429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:44.936703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:44.936976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:45.037844 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:45.039944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:45.052465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1075/2153] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:53.961463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:30:53.962870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:30:53.974751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e23d2c-6223-4d99-94b9-1ffff9a9b873 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:30:53.978835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:30:53.979445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:30:53.979746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:30:53.980016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:30:53.989705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:30:53.991156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46564bcd-8acf-4ae9-8a60-54b6ebc40bf9 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:30:54.000819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1079/2154] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:30:55.045471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:00.137948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:00.139470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.203864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ae768-22e2-4e43-9557-5f9131198e24 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:00.208860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:00.209477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.209747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.210015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] PATH_INFO: `/v3/users/5119931234d248e3aefeec7904598680` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.211130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5119931234d248e3aefeec7904598680)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:00.213055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:00.260896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5119931234d248e3aefeec7904598680'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.267808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5119931234d248e3aefeec7904598680'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.280234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5119931234d248e3aefeec7904598680'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.280883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5119931234d248e3aefeec7904598680'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.288207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5119931234d248e3aefeec7904598680'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.294083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1076/2155] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:00 2022] DELETE /identity/v3/users/5119931234d248e3aefeec7904598680 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:00.298486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.298937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.299043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.317374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:00.318552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.376019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1080/2156] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:00.386202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:00.387629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.444200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d6be-ebdf-4e6d-8275-32cf29d658fc admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:00.447912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:00.448545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.448826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.449096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.459192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:00.460652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5c0cf-9a3a-4ac5-b897-69d78d06cfe6 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:00.470116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1077/2157] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:00.748146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:00.751016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.763297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8f8d8-3260-444e-bd4d-d98cff7551bf None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:00.769005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:00.769761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.770146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.771211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] PATH_INFO: `/v3/projects/299f1a0143cc4e99a5402f020640bae0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.772597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=299f1a0143cc4e99a5402f020640bae0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:00.774987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:00.776812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:00.839138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '299f1a0143cc4e99a5402f020640bae0', 'request_id': 'req-92535936-ed35-448a-915c-29bd1600be92'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.856467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '299f1a0143cc4e99a5402f020640bae0', 'request_id': 'req-92535936-ed35-448a-915c-29bd1600be92'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.856467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '299f1a0143cc4e99a5402f020640bae0', 'request_id': 'req-92535936-ed35-448a-915c-29bd1600be92'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.866706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] The token cache is being invalidate because project 299f1a0143cc4e99a5402f020640bae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:00.866990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.867669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-92535936-ed35-448a-915c-29bd1600be92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:00.869679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1081/2158] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:00 2022] DELETE /identity/v3/projects/299f1a0143cc4e99a5402f020640bae0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:00.874666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.874950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.875233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.896226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:00.897444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.914667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:00.959833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1078/2159] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:00.967383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:00.967668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:00.967938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:00.987726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:00.988946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:00.996804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1082/2160] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:01.000832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.011127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc1fa3-6aa5-44e7-a162-3525b3726caa None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.015618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.016203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.016508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.016775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.017402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:31:01.017721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.019303 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.030422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1079/2161] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:31:01.034519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.043569 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d111b-f7cb-4149-aa36-22ab818eb28d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.048118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.048742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.049016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.049782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.050399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.052267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.079057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1083/2162] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:01 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:01.083233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.095312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b39e4a-6903-4fd0-a939-30bc8ad1b6b9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.154323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.155046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.155335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.155610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.156251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.158185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.180522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1080/2163] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:01 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:31:01.184778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.194678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7d6ef-ae82-49c2-83a2-7b995fb459a1 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.199243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.199832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.200105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.200412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.201025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.202603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.212833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1084/2164] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:31:01.216571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.227800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a0573-fac7-4ada-83e3-39fe75cf6573 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.232673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.233279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.233559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.233829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.234494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.236158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088f606-5a7b-4ff5-b1ab-d3aa2e460c01 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.247690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1081/2165] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:31:01.251870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.262052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeeb32-e00e-47f9-b8d1-63cfbe8e1200 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.267206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.267864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.268203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.268515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] PATH_INFO: `/v3/projects/2994659c171e4836a7a2f25137dc18b6/users/183ee46b6a3441a5a288322effed25ea/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.283038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2994659c171e4836a7a2f25137dc18b6, user_id=183ee46b6a3441a5a288322effed25ea, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.286265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.299027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1085/2166] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:01 2022] PUT /identity/v3/projects/2994659c171e4836a7a2f25137dc18b6/users/183ee46b6a3441a5a288322effed25ea/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:01.310934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.311216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.311492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.338632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] MFA Rules not processed for user `183ee46b6a3441a5a288322effed25ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:01.339829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:01.404599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1082/2167] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:01.415789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:01.417531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:01.472542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524b33c-b662-4fcc-9b74-fcbc2e0ef8a8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:01.476007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:01.476623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:01.476900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:01.477426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:01.485563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:01.487603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36dd57a-bb37-490c-be1e-fe22713d7428 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:01.496128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1086/2168] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:07.844884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:07.865618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:07.910393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b43200-df83-4c7e-8071-1f16fe81859b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:07.914825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:07.915439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:07.915719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:07.915991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] PATH_INFO: `/v3/users/183ee46b6a3441a5a288322effed25ea` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:07.917138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=183ee46b6a3441a5a288322effed25ea)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:07.919044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:07.964503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183ee46b6a3441a5a288322effed25ea'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:07.971249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183ee46b6a3441a5a288322effed25ea'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:07.982897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183ee46b6a3441a5a288322effed25ea'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:07.983510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183ee46b6a3441a5a288322effed25ea'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:07.992667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183ee46b6a3441a5a288322effed25ea'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:07.998364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1083/2169] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:07 2022] DELETE /identity/v3/users/183ee46b6a3441a5a288322effed25ea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:08.002714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.003686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.003798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.022753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:08.023918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:08.046929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:08.084809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1087/2170] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:08.094169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.095668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:08.154159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735301c-b08f-4e4b-bf36-f9fd36fca92e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.157795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.158389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.158661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.158926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.168396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.169851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d52182-2c9f-4f6f-95f1-a0d14b8dcb3b service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.179351 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1084/2171] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:08.471879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.473438 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:08.486554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9505ef-28fb-4c9d-bedc-b583cc3c91c6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.492883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.493700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.494099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.494430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] PATH_INFO: `/v3/projects/2994659c171e4836a7a2f25137dc18b6` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.496288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2994659c171e4836a7a2f25137dc18b6)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.498232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.545093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2994659c171e4836a7a2f25137dc18b6', 'request_id': 'req-fa3f9f50-0896-499b-bb7d-48400affebf8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:08.553084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2994659c171e4836a7a2f25137dc18b6', 'request_id': 'req-fa3f9f50-0896-499b-bb7d-48400affebf8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:08.558680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2994659c171e4836a7a2f25137dc18b6', 'request_id': 'req-fa3f9f50-0896-499b-bb7d-48400affebf8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:08.570238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] The token cache is being invalidate because project 2994659c171e4836a7a2f25137dc18b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:08.570527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:08.571166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fa3f9f50-0896-499b-bb7d-48400affebf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:08.573813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1088/2172] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:08 2022] DELETE /identity/v3/projects/2994659c171e4836a7a2f25137dc18b6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:08.578600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.578944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.579255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.599141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:08.600348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:08.634204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:08.661499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1085/2173] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:08.669193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.669476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.669745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.691432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:08.692757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:08.700851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1089/2174] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:08.704848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.715566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334e7b4-74e2-4e0d-b411-07251ec8aa87 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.720163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.720802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.721083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.721357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.722503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:31:08.722807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.724433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.736785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1086/2175] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:31:08.741698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.752983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1b9a3-4d64-4d10-8383-b4a71bc72807 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.759375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.759983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.760253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.760549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.761355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.763091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.794723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1090/2176] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:08 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:31:08.799037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.809505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b723bfb-641a-43d0-81fb-e37ebf479e2d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.870909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.871551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.871874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.872153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.872818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.874941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.895785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1087/2177] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:08 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:08.899839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.910433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96e816-2824-49a8-8ad9-787f223dda0f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.915075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.915660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.915935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.916204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.916877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.918450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.929204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1091/2178] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:31:08.932888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.942629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b2d3-a89f-4abc-bb4f-ca9e2288015d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.947175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.947763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.948039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.948328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.948962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.950523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1742ae1-0d17-4856-8234-752211e7088d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:08.960725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1088/2179] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:08.964796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:08.975238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231be15-aa54-4624-8b66-85b9800d5456 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:08.980071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:08.980742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:08.981018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:08.981288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] PATH_INFO: `/v3/projects/0143fc569a0f43e79149474a7b604e53/users/5256c9d440a643a197e0df1876023e28/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:08.995200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0143fc569a0f43e79149474a7b604e53, user_id=5256c9d440a643a197e0df1876023e28, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:08.997856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.010600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1092/2180] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:08 2022] PUT /identity/v3/projects/0143fc569a0f43e79149474a7b604e53/users/5256c9d440a643a197e0df1876023e28/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:09.021444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.021783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.022064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.050177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] MFA Rules not processed for user `5256c9d440a643a197e0df1876023e28`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:09.051377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:09.113432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1089/2181] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:09.117831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.165332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:09.179227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31f083-6930-436c-91fa-10343139438a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.183843 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.184466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.184746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.185016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.185639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.187527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.212709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1093/2182] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:09.216799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.228056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7a231-ce98-4ce4-a455-5def46953d96 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.261484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:09.287849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.288479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.288761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.289035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.289653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.291593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.316188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1090/2183] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:09.317402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.328868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cc6d0-2a31-4d63-9cb7-8707e6634567 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.334096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.334690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.334969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.335241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.335871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.337483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.348673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1094/2184] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:09.353016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.365886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f7e39-d204-4b07-9655-3b48387eff6a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.370602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.371220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.371492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.371758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.372423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.373984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600861d-23e5-4bf5-af22-93a34555d884 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.384670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1091/2185] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:09.388824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.400136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f156-2eac-4e16-846e-78af1a8fec2d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.404948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.405601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.405877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.406145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] PATH_INFO: `/v3/projects/0099acccd8d74358b7478158af7a41b2/users/167ffc1c501f44d2b433a29bc8c82463/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.421490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0099acccd8d74358b7478158af7a41b2, user_id=167ffc1c501f44d2b433a29bc8c82463, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.424028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.440327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1095/2186] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:09 2022] PUT /identity/v3/projects/0099acccd8d74358b7478158af7a41b2/users/167ffc1c501f44d2b433a29bc8c82463/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:09.508255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.508597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.508862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.536518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] MFA Rules not processed for user `167ffc1c501f44d2b433a29bc8c82463`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:09.537725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:09.604163 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1092/2187] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:09.605273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.665848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5745bee-b310-4832-9dd1-91b01118b2d9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.670383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.670970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.671242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.671559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.672135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.674036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.699906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1096/2188] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:09.703898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.715004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c92b2-c2a1-4371-a366-24b57217b67a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.769681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.770279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.770557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.770827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.771449 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.773363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.793365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1093/2189] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:09 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:09.797425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.808098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a4627-5fa3-48fb-b969-a5913b0b14fe admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.813020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.813610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.813887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.814157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.814787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.817075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.827461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1097/2190] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:09.831754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.841767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ffa4d-edeb-4af6-ab42-7db80ad43949 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.846558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.847459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.847739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.848010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] PATH_INFO: `/v3/projects/363ddf8200834eefb8432f3687226af0/users/34109c35584b4763ac40faaf19a476b5/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.865141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363ddf8200834eefb8432f3687226af0, user_id=34109c35584b4763ac40faaf19a476b5, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.873052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.886089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1094/2191] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:09 2022] PUT /identity/v3/projects/363ddf8200834eefb8432f3687226af0/users/34109c35584b4763ac40faaf19a476b5/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:09.889844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.899701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e82d6-beaf-4f1f-9ac7-4d0aea508623 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.957007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.957615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.957955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.958175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.958816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:09.960439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92abe085-38d8-483f-b78a-6c1b1ff08094 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:09.971115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1098/2192] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:09.975469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:09.989550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eaff9-9c97-49c9-afad-16a6e1d11ed3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:09.994309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:09.994965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:09.995270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:09.995547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] PATH_INFO: `/v3/projects/363ddf8200834eefb8432f3687226af0/users/34109c35584b4763ac40faaf19a476b5/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:09.997514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=363ddf8200834eefb8432f3687226af0, user_id=34109c35584b4763ac40faaf19a476b5, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:10.000129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:10.004016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:10.012600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1095/2193] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:09 2022] PUT /identity/v3/projects/363ddf8200834eefb8432f3687226af0/users/34109c35584b4763ac40faaf19a476b5/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:31:10.022823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:10.023381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:10.023381 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:10.042601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:10.052061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] MFA Rules not processed for user `34109c35584b4763ac40faaf19a476b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:10.053755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:10.120891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1099/2194] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:10.131918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:10.133601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:10.189739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90f679-4965-4025-9a2d-63b716c4aef1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:10.193366 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:10.193954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:10.194231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:10.194504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:10.249635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:10.250985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:10.259913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1096/2195] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:11.796068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:11.798350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:11.831443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed210ac-1420-4c17-a51a-5d7efd651658 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:11.843092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:11.843680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:11.843954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:11.844436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:11.957712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:11.959352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:11.971509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1100/2196] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:12.566875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:12.575482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8359707-a151-4fd6-b4fb-0a4f25c882b9 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:12.579356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:12.580034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:12.580530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:12.580930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:12.590836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:12.592288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2e2d8c-595b-48e2-8856-8c891e3e990f service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:12.601742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1097/2197] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:17.153207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:17.154658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:17.222574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d40c0-36ea-4f50-a808-06428943eb79 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:17.227386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:17.228016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:17.228311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:17.228588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] PATH_INFO: `/v3/users/1ff146e8a8ec4490a4917b033e481440` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:17.229700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff146e8a8ec4490a4917b033e481440)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:17.231620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:17.276519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:17.289350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff146e8a8ec4490a4917b033e481440'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:17.296828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff146e8a8ec4490a4917b033e481440'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:17.297464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff146e8a8ec4490a4917b033e481440'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:17.304517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff146e8a8ec4490a4917b033e481440'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:17.317190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff146e8a8ec4490a4917b033e481440'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:17.323066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1101/2198] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:17 2022] DELETE /identity/v3/users/1ff146e8a8ec4490a4917b033e481440 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:17.327800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:17.328274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:17.328386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:17.348402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:17.349598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:17.443899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1098/2199] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:17.453735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:17.455291 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:17.555286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5e883-e067-451f-a14f-e4315c243486 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:17.559297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:17.560131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:17.561102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:17.561372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:17.573496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:17.574981 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2a1a9-37d5-4dbe-ba40-f0edaea48ad0 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:17.589157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1102/2200] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:17.946166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:17.947724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:17.961421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954434b-f273-4744-98bd-9ab4bfe374dc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:17.968036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:17.969030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:17.969544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:17.970023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] PATH_INFO: `/v3/projects/02428e22c6704c5dbe2021df863b9cd5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:17.971659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] RBAC: Authorizing `identity:delete_project(project_id=02428e22c6704c5dbe2021df863b9cd5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:17.974342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.022569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:18.035916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02428e22c6704c5dbe2021df863b9cd5', 'request_id': 'req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:18.045965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02428e22c6704c5dbe2021df863b9cd5', 'request_id': 'req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:18.057837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02428e22c6704c5dbe2021df863b9cd5', 'request_id': 'req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:18.070190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] The token cache is being invalidate because project 02428e22c6704c5dbe2021df863b9cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:18.070473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:18.071121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5f5a4a8e-2c55-42ce-9f37-f1f94447986e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:18.072616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1099/2201] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:17 2022] DELETE /identity/v3/projects/02428e22c6704c5dbe2021df863b9cd5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:18.077447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.077730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.077997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.097074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:18.098266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:18.156045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1103/2202] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:18.164793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.165077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.165349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.185071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:18.186300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:18.194544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1100/2203] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:18.198594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.208099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f7b55-aa22-4ccf-abd6-a849ae702f1e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.212880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.213474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.213752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.214023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.214657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:31:18.214954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.216634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.228228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1104/2204] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:31:18.232257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.241644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75da1de-ed2c-42a6-9934-1941c98002cb None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.246207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.246782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.247052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.247316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.247934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.249825 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.276405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1101/2205] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:18 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:31:18.280430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.290161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3eb20-bf36-454a-afbb-7ecfe4df807e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.329271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:18.346374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.347477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.347754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.348018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.348669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.350558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.371817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1105/2206] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:18 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:18.375794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.386323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee26a38-7647-449c-8052-de7d38a7d51f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.390931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.391522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.391806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.392078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.392751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.394335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.404028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1102/2207] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:18.407902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.418193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda0402-b077-4b82-bb25-7e393af56e31 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.422864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.423445 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.423721 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.423993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.424655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.426829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047fe74-3363-4815-94d0-66a9d7ccc7b9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.437129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1106/2208] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:18.441042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.450831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda73546-522a-460c-998f-e39e9597fc24 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.455367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.456013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.456311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.456602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] PATH_INFO: `/v3/projects/64ab512913704ae1b116cd0c3189760b/users/0f090f1b5d1449fca5f077b6f6a9be1f/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.470596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ab512913704ae1b116cd0c3189760b, user_id=0f090f1b5d1449fca5f077b6f6a9be1f, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.473792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.486743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1103/2209] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:18 2022] PUT /identity/v3/projects/64ab512913704ae1b116cd0c3189760b/users/0f090f1b5d1449fca5f077b6f6a9be1f/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:18.497365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.497651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.497923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.525150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] MFA Rules not processed for user `0f090f1b5d1449fca5f077b6f6a9be1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:18.526352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:18.588466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1107/2210] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:18.599604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.601319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:18.658558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f5930-5ca4-498e-a0cf-8098e1b5ef6e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.663490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.664089 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.664409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.664682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.673258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.674744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca885c52-3308-4095-b6d1-fbe6af7de841 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:18.684439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1104/2211] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:31:18.897248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:18.899092 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:18.981454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a2137-384f-45c5-a5ba-c058b46ca49e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:18.986172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:18.986825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:18.987329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:18.987578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] PATH_INFO: `/v3/users/5256c9d440a643a197e0df1876023e28` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:18.989053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5256c9d440a643a197e0df1876023e28)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:18.990980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:19.085152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5256c9d440a643a197e0df1876023e28'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.106510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:19.114156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5256c9d440a643a197e0df1876023e28'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.116269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5256c9d440a643a197e0df1876023e28'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.127375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5256c9d440a643a197e0df1876023e28'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.146427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5256c9d440a643a197e0df1876023e28'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.153384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1108/2212] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:18 2022] DELETE /identity/v3/users/5256c9d440a643a197e0df1876023e28 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:19.157925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:19.158321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:19.158724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:19.181018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:19.181962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:19.242004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:19.252419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1105/2213] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:19.267914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:19.270081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:19.329607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380255d-a1eb-41b9-adfa-709a907b74fb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:19.334001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:19.334601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:19.334878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:19.335144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:19.344208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:19.347608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:19.355709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1109/2214] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:19.651216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:19.652946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:19.685424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e31ca-2287-4082-9e29-26d10c3dc2cb None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:19.691655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:19.692376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:19.692719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:19.693159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] PATH_INFO: `/v3/projects/0143fc569a0f43e79149474a7b604e53` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:19.694492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0143fc569a0f43e79149474a7b604e53)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:19.696426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:19.749904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0143fc569a0f43e79149474a7b604e53', 'request_id': 'req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.758710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0143fc569a0f43e79149474a7b604e53', 'request_id': 'req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.764396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0143fc569a0f43e79149474a7b604e53', 'request_id': 'req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.775791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] The token cache is being invalidate because project 0143fc569a0f43e79149474a7b604e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:19.776079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.776776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-86b7cc7b-f16e-4b84-b632-865dbcb5f13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.778288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1106/2215] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:19 2022] DELETE /identity/v3/projects/0143fc569a0f43e79149474a7b604e53 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:19.782198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:19.792049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b11411-1bca-4a3b-b4c6-647df5f8440a service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:19.850927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:19.851560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:19.851833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:19.852352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] PATH_INFO: `/v3/users/167ffc1c501f44d2b433a29bc8c82463` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:19.853466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167ffc1c501f44d2b433a29bc8c82463)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:19.855378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:19.903829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167ffc1c501f44d2b433a29bc8c82463'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.911902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167ffc1c501f44d2b433a29bc8c82463'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.912537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167ffc1c501f44d2b433a29bc8c82463'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.919581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167ffc1c501f44d2b433a29bc8c82463'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.928679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:19.933581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167ffc1c501f44d2b433a29bc8c82463'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:19.941132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1110/2216] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:19 2022] DELETE /identity/v3/users/167ffc1c501f44d2b433a29bc8c82463 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:20.164521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:20.164984 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:20.165430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:20.185514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] MFA Rules not processed for user `0f090f1b5d1449fca5f077b6f6a9be1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:20.186959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:20.220014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:20.222330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:20.278813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1107/2217] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:20.305945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a18d1-db08-4406-ab66-b41351a3eeb6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:20.314058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:20.315597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:20.315876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:20.316145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] PATH_INFO: `/v3/projects/0099acccd8d74358b7478158af7a41b2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:20.317468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0099acccd8d74358b7478158af7a41b2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:20.321185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:20.397154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0099acccd8d74358b7478158af7a41b2', 'request_id': 'req-7fd3d277-63f7-406e-9577-49d6c7d0a85c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.406519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0099acccd8d74358b7478158af7a41b2', 'request_id': 'req-7fd3d277-63f7-406e-9577-49d6c7d0a85c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.414032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0099acccd8d74358b7478158af7a41b2', 'request_id': 'req-7fd3d277-63f7-406e-9577-49d6c7d0a85c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.428762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] The token cache is being invalidate because project 0099acccd8d74358b7478158af7a41b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:20.429096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.429816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.431517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1111/2218] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:20 2022] DELETE /identity/v3/projects/0099acccd8d74358b7478158af7a41b2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:31:20.441649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:20.452679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480da31-1645-40f1-b669-1d7ebadb7bb1 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:20.514101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:20.514759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:20.515112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:20.515396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] PATH_INFO: `/v3/users/34109c35584b4763ac40faaf19a476b5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:20.516704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34109c35584b4763ac40faaf19a476b5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:20.519239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:20.557851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:20.576953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34109c35584b4763ac40faaf19a476b5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.585883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34109c35584b4763ac40faaf19a476b5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.604503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34109c35584b4763ac40faaf19a476b5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.605403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34109c35584b4763ac40faaf19a476b5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.616737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34109c35584b4763ac40faaf19a476b5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:20.623291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1108/2219] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:20 2022] DELETE /identity/v3/users/34109c35584b4763ac40faaf19a476b5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:21.012533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:21.014324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:21.034767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3d277-63f7-406e-9577-49d6c7d0a85c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:21.097900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:21.098795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:21.099199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:21.099509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] PATH_INFO: `/v3/projects/363ddf8200834eefb8432f3687226af0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:21.100866 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363ddf8200834eefb8432f3687226af0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:21.103126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:21.129898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:21.155659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363ddf8200834eefb8432f3687226af0', 'request_id': 'req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.162803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363ddf8200834eefb8432f3687226af0', 'request_id': 'req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.169377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363ddf8200834eefb8432f3687226af0', 'request_id': 'req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.181397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] The token cache is being invalidate because project 363ddf8200834eefb8432f3687226af0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:21.181773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.182446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ca5a541-7543-4ac0-9e05-e2d0e249dc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.183939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1112/2220] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:21 2022] DELETE /identity/v3/projects/363ddf8200834eefb8432f3687226af0 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:21.467210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:21.468645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:21.481901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37bd17-d64f-43bb-b9a7-9860f785c547 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:21.536728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:21.537255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:21.537536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:21.537806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] PATH_INFO: `/v3/users/75a0e5b3262b4f399c1ea55ff4b354d9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:21.538906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a0e5b3262b4f399c1ea55ff4b354d9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:21.540864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:21.585222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a0e5b3262b4f399c1ea55ff4b354d9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.592059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a0e5b3262b4f399c1ea55ff4b354d9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.604664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a0e5b3262b4f399c1ea55ff4b354d9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.605294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a0e5b3262b4f399c1ea55ff4b354d9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.613098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a0e5b3262b4f399c1ea55ff4b354d9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:21.618929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1109/2221] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:21 2022] DELETE /identity/v3/users/75a0e5b3262b4f399c1ea55ff4b354d9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:21.623385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:21.623837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:21.623942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:21.642499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:21.643716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:21.700220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1113/2222] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:21.710508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:21.711957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:21.742747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:21.770783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d71bf4-df91-4b2f-b8ab-20ec09e1a992 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:21.774464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:21.775066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:21.775345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:21.775618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:21.785354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:21.786828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:21.796515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1110/2223] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:22.178707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:22.180653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:22.197545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3393b-c429-469e-913f-81381256c3c4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:22.213898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:22.214797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:22.215904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:22.216392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] PATH_INFO: `/v3/projects/df150e1afc044ef0b8479ccb7ca8dfc5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:22.218145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] RBAC: Authorizing `identity:delete_project(project_id=df150e1afc044ef0b8479ccb7ca8dfc5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:22.228841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:22.297853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df150e1afc044ef0b8479ccb7ca8dfc5', 'request_id': 'req-332e21ab-aced-4135-aaf9-0d835ee2b16a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.308192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df150e1afc044ef0b8479ccb7ca8dfc5', 'request_id': 'req-332e21ab-aced-4135-aaf9-0d835ee2b16a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.315185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df150e1afc044ef0b8479ccb7ca8dfc5', 'request_id': 'req-332e21ab-aced-4135-aaf9-0d835ee2b16a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.335620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] The token cache is being invalidate because project df150e1afc044ef0b8479ccb7ca8dfc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:22.335916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.337250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.338974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1114/2224] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:22 2022] DELETE /identity/v3/projects/df150e1afc044ef0b8479ccb7ca8dfc5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:22.342977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:22.353655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dec72-f4e0-4228-bfec-967e97e5c24b service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:22.419725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:22.420370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:22.420648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:22.420921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] PATH_INFO: `/v3/users/05139d3dc6494135b999f7d290639278` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:22.422010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=05139d3dc6494135b999f7d290639278)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:22.424096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:22.472274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05139d3dc6494135b999f7d290639278'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.480174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05139d3dc6494135b999f7d290639278'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.494884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05139d3dc6494135b999f7d290639278'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.496073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05139d3dc6494135b999f7d290639278'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.505464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-6f1c8db4-b8e7-49f9-82e7-97656a94b8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05139d3dc6494135b999f7d290639278'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.511328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1111/2225] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:22 2022] DELETE /identity/v3/users/05139d3dc6494135b999f7d290639278 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:22.773692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:22.775179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:22.788698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e21ab-aced-4135-aaf9-0d835ee2b16a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:22.822184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:22.850957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:22.851589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:22.851870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:22.852448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] PATH_INFO: `/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:22.853751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2616ed3d543a4031a8ba3457dbcbe0f7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:22.855667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:22.902332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2616ed3d543a4031a8ba3457dbcbe0f7', 'request_id': 'req-3b0aeb55-14f1-4f26-aad3-10ef130a2333'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.908723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2616ed3d543a4031a8ba3457dbcbe0f7', 'request_id': 'req-3b0aeb55-14f1-4f26-aad3-10ef130a2333'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.914745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2616ed3d543a4031a8ba3457dbcbe0f7', 'request_id': 'req-3b0aeb55-14f1-4f26-aad3-10ef130a2333'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.926288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] The token cache is being invalidate because project 2616ed3d543a4031a8ba3457dbcbe0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:22.926574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.927206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b0aeb55-14f1-4f26-aad3-10ef130a2333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:22.928708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1115/2226] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:22 2022] DELETE /identity/v3/projects/2616ed3d543a4031a8ba3457dbcbe0f7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:22.935277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:22.935635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:22.935914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:22.955047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:22.956252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:23.000698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:23.022036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1112/2227] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:23.029969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.030249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.030519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.051200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:23.052490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:23.060339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1116/2228] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:23.064620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.075903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3fea4-62ad-4b1d-969d-e286b5bf7567 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.080660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.081392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.081664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.081926 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.082567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:31:23.082865 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.084488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.102046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1113/2229] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:31:23.107877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.119230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaf68d-4221-4659-97c2-2261bfb34a10 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.124132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.124776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.125054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.125326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.125950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.127882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.164346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1117/2230] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:23 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:23.167130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.178976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262dfbc0-942e-4c43-aaef-b8229a35b756 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.240906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.241536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.241812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.242081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.242705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.244628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.271669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1114/2231] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:23 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:23.275304 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.286128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32e9d6-45a2-426d-ad3d-881e66b172a2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.291008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.291626 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.291906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.292349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.293314 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.294614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.309762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1118/2232] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:23.310917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.321410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea118cc-22f0-411f-b3af-c248c3040b2d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.327298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.327897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.328170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.328472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.329110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.331530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5c66a0-a400-44a7-a469-9ea55857aefb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.350748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1115/2233] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:31:23.355053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.367592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d28a07a-89dc-448b-ae0a-31342f5c4fde admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.372462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.373261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.373546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.373815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] PATH_INFO: `/v3/projects/543e3a7b5af24948a0ad0519d000e209/users/70d839834f5240dd91c5f7f615ffe36f/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.390284 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=543e3a7b5af24948a0ad0519d000e209, user_id=70d839834f5240dd91c5f7f615ffe36f, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.393003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.406042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1119/2234] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:23 2022] PUT /identity/v3/projects/543e3a7b5af24948a0ad0519d000e209/users/70d839834f5240dd91c5f7f615ffe36f/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:23.411993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d93378-47e8-4606-bae1-51dfceb03485 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.412374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d93378-47e8-4606-bae1-51dfceb03485 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.412646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d93378-47e8-4606-bae1-51dfceb03485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.431960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a8d93378-47e8-4606-bae1-51dfceb03485 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:23.433173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a8d93378-47e8-4606-bae1-51dfceb03485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:23.504670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1116/2235] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:23.518252 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:23.521507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:23.595458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f600d0-7c41-47f8-92c9-48cf93353975 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:23.603714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:23.604559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:23.604951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:23.605315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:23.622802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:23.624268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:23.644635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1120/2236] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:24.459662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59b155-0687-4fa5-a291-b6de3976c0a2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:24.459969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59b155-0687-4fa5-a291-b6de3976c0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:24.460244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59b155-0687-4fa5-a291-b6de3976c0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:24.507698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1e59b155-0687-4fa5-a291-b6de3976c0a2 None None] MFA Rules not processed for user `70d839834f5240dd91c5f7f615ffe36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:24.509744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1e59b155-0687-4fa5-a291-b6de3976c0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:24.559635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:24.587972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1117/2237] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:24.598970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:24.600508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:24.607512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:24.701917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d657d-1ae3-4e0f-8e48-7c25a853fa24 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:24.707559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:24.708648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:24.708978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:24.709256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:24.719724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:24.722244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274c009-762f-4434-af53-5719ee4be1de service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:24.735558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1121/2238] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:26.179393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e65e-2726-4e57-b43c-d18c50f60ff7 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:26.179925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e65e-2726-4e57-b43c-d18c50f60ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:26.179925 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e65e-2726-4e57-b43c-d18c50f60ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:26.199299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e963e65e-2726-4e57-b43c-d18c50f60ff7 None None] MFA Rules not processed for user `70d839834f5240dd91c5f7f615ffe36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:26.200790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e963e65e-2726-4e57-b43c-d18c50f60ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:26.209082 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1118/2239] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:26.416374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e6c60-78bb-46d9-acd2-dca58be9a845 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:26.416882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e6c60-78bb-46d9-acd2-dca58be9a845 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:26.416992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e6c60-78bb-46d9-acd2-dca58be9a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:26.438254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c31e6c60-78bb-46d9-acd2-dca58be9a845 None None] MFA Rules not processed for user `0847f00afbde44068fcf1cedbce24be5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:26.439472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c31e6c60-78bb-46d9-acd2-dca58be9a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:26.498627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1122/2240] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:32.982819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:32.983547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:32.983805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:33.025530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] MFA Rules not processed for user `55a7f7ba0b27479f8d1e68d808637d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:33.026842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:33.088019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1119/2241] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:36.491497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=49 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:43.081288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=35 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:45.424871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9366004-4fa0-4da5-a623-2dd42010bd60 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:45.425901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9366004-4fa0-4da5-a623-2dd42010bd60 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:45.426386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9366004-4fa0-4da5-a623-2dd42010bd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:45.445508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f9366004-4fa0-4da5-a623-2dd42010bd60 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:45.446711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f9366004-4fa0-4da5-a623-2dd42010bd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:45.506391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1123/2242] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:45.548690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:45.550773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:45.629651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120b5fde-bf1d-4daa-9820-b981b5635a9a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:45.634828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:45.635769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:45.636059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:45.636377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:45.647187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:45.649511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e3efa-483b-44a5-92ba-6c8ddeec8e15 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:45.664498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1120/2243] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:31:45.695555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:45.695840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:45.696111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:45.720164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] MFA Rules not processed for user `70d839834f5240dd91c5f7f615ffe36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:45.721943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:45.729929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1124/2244] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:46.263233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1296c9-d2c9-4859-a35b-23a2c7ef0e3c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:46.263549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1296c9-d2c9-4859-a35b-23a2c7ef0e3c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:46.263826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1296c9-d2c9-4859-a35b-23a2c7ef0e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:46.283368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7f1296c9-d2c9-4859-a35b-23a2c7ef0e3c None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:46.284575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7f1296c9-d2c9-4859-a35b-23a2c7ef0e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:46.292287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1121/2245] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:46.305983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:46.318691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366b87-b121-48f0-9438-a2d1a9f31ce8 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:46.324185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:46.324963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:46.325444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:46.326062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:46.335327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:46.337200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:46.345766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1125/2246] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:31:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:31:46.635373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04a35-d2ef-4121-b31a-bb1401a1a233 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:46.635747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04a35-d2ef-4121-b31a-bb1401a1a233 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:46.635949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04a35-d2ef-4121-b31a-bb1401a1a233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:46.656463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-02f04a35-d2ef-4121-b31a-bb1401a1a233 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:46.657999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-02f04a35-d2ef-4121-b31a-bb1401a1a233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:46.668028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1122/2247] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:31:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:46.693037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:46.704639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af141383-1f2b-48f9-af73-0a6ee58bbc12 service swift] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:46.710799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:46.711432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:46.711720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:46.711973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:46.721598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:46.723331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:46.733480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1126/2248] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:31:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:31:47.192195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:47.192793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:47.193161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:47.214910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:47.217078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:47.230617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1123/2249] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:31:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:47.246825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:47.267207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3b14b-a3e2-4207-99e3-e80f751fc762 service swift] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:47.273273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:47.273949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:47.274230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:47.274502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:47.284586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:47.287170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e627d6e-98f5-4c53-9b56-ff443ba2a8a4 service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:47.304435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1127/2250] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:31:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:31:49.768598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:49.770306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:49.785612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0370347-0680-4018-bb4a-7f82f5166067 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:49.792127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:49.792834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:49.793096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:49.793577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] PATH_INFO: `/v3/users/0f090f1b5d1449fca5f077b6f6a9be1f` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:49.794765 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f090f1b5d1449fca5f077b6f6a9be1f)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:49.796756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:49.849039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f090f1b5d1449fca5f077b6f6a9be1f'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:49.858239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f090f1b5d1449fca5f077b6f6a9be1f'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:49.885333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f090f1b5d1449fca5f077b6f6a9be1f'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:49.885537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f090f1b5d1449fca5f077b6f6a9be1f'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:49.893429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f090f1b5d1449fca5f077b6f6a9be1f'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:49.899670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1124/2251] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:31:49 2022] DELETE /identity/v3/users/0f090f1b5d1449fca5f077b6f6a9be1f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:31:49.904119 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:49.904630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:49.904745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:49.930773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:49.932062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:50.024015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1128/2252] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:50.033484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.035327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:50.092761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:50.119976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd1c53-9a3a-441e-9d8b-0879e9a80895 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.131682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.133677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.134052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.137736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.157214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.158701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6162df21-01d7-4c98-87df-4c25a2932268 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.191598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1125/2253] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:50.402967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.405061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:50.419682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39862a2a-ab46-4bfd-9898-64c1e1fe680a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.427220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.428159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.428996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.429500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] PATH_INFO: `/v3/projects/64ab512913704ae1b116cd0c3189760b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.432351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ab512913704ae1b116cd0c3189760b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.436044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.495267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ab512913704ae1b116cd0c3189760b', 'request_id': 'req-0ea096a0-bccc-490c-910a-bec52f4158ae'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:50.499287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:50.504914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ab512913704ae1b116cd0c3189760b', 'request_id': 'req-0ea096a0-bccc-490c-910a-bec52f4158ae'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:50.512779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ab512913704ae1b116cd0c3189760b', 'request_id': 'req-0ea096a0-bccc-490c-910a-bec52f4158ae'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:50.524583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] The token cache is being invalidate because project 64ab512913704ae1b116cd0c3189760b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:31:50.524955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:50.525677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0ea096a0-bccc-490c-910a-bec52f4158ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:31:50.527337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1129/2254] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:31:50 2022] DELETE /identity/v3/projects/64ab512913704ae1b116cd0c3189760b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:31:50.532114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.532495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.532864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.553024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:50.554127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:50.615150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1126/2255] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:50.623345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.623775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.624194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.645656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:50.646985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:50.655041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1130/2256] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:50.659788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.669294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed69e9-86d2-413f-a86f-6e779c01c49b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.673788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.675211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.675492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.675756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.676411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:31:50.676709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.678291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.692341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1127/2257] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:31:50.694374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.703476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a5f29-7191-4347-810b-64dc9c6f9f15 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.708180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.708824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.709129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.709397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.710014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.711944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.743353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1131/2258] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:31:50 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:50.744520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.754549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58248ccb-47a4-4024-b1d0-b8c885f3daf8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.824482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.825090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.825363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.825632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.826258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.828158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.854531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1128/2259] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:31:50 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:31:50.856627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.867551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c9811-beff-44fc-a1cf-26e8088465b9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.876145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.876780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.877060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.877332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.878034 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.879643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.889856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1132/2260] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:50.893847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.903901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44716d-badb-49a1-8ee7-23caab3262c4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.909016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.909599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.909870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.910134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.910760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.912338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2905fd9-2aef-44de-bff2-f0bc61b919fb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.923329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1129/2261] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:31:50.927997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:50.937781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a956c07-8cc9-4802-a338-a8aa31c25248 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:50.942367 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:50.942995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.943268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.943560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] PATH_INFO: `/v3/projects/24db83f7bc4d4888b73d4d4ab7c4bdce/users/44c2913f48ad4200b6baf14c0830770c/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.958497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=24db83f7bc4d4888b73d4d4ab7c4bdce, user_id=44c2913f48ad4200b6baf14c0830770c, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:50.961196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:50.975416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1133/2262] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:31:50 2022] PUT /identity/v3/projects/24db83f7bc4d4888b73d4d4ab7c4bdce/users/44c2913f48ad4200b6baf14c0830770c/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:31:50.980615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d1a08-971d-4567-82c7-07cb68dad808 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:50.980898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d1a08-971d-4567-82c7-07cb68dad808 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:50.981172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7d1a08-971d-4567-82c7-07cb68dad808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:50.999498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4d7d1a08-971d-4567-82c7-07cb68dad808 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:51.000716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4d7d1a08-971d-4567-82c7-07cb68dad808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:51.047864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:31:51.064400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1130/2263] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:31:51.073905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:51.075585 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:51.132572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd124c9d-0549-4e69-8cc8-fb016bd41adf admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:51.136286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:51.136912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:51.137193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:51.137462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:51.147101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:51.148546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:51.158076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1134/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:51.866197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bc64-9673-4958-a7c4-342e66f577a1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:51.866675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bc64-9673-4958-a7c4-342e66f577a1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:51.868573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957bc64-9673-4958-a7c4-342e66f577a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:51.903187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1957bc64-9673-4958-a7c4-342e66f577a1 None None] MFA Rules not processed for user `44c2913f48ad4200b6baf14c0830770c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:51.904899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1957bc64-9673-4958-a7c4-342e66f577a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:51.983492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1131/2265] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:31:51.997529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:31:52.000068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:52.067351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151d8d-63d4-4c0e-b938-e62100dfbd45 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:31:52.070951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:31:52.071547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:52.071822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:52.072091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:52.080189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:31:52.081570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4381ae08-950a-4560-8b69-608d631329aa service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:31:52.090086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1135/2266] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:31:53.459786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:31:53.460363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:31:53.460363 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:31:53.480288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] MFA Rules not processed for user `44c2913f48ad4200b6baf14c0830770c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:31:53.481552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:31:53.490308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1132/2267] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:31:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:02.086243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=85,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:03.475060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=39 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:09.508480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c55d4-671f-42a3-ac60-dc3817ba4135 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:09.508950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c55d4-671f-42a3-ac60-dc3817ba4135 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:09.509039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c55d4-671f-42a3-ac60-dc3817ba4135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:09.528789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-fe9c55d4-671f-42a3-ac60-dc3817ba4135 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:09.529662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fe9c55d4-671f-42a3-ac60-dc3817ba4135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:09.586739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1136/2268] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:09.599980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:09.601390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:09.636182 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59561a2-b4f2-4668-a8ef-7fb31270cb94 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:09.636577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59561a2-b4f2-4668-a8ef-7fb31270cb94 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:09.636848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59561a2-b4f2-4668-a8ef-7fb31270cb94 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:09.638173 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1137/2269] 10.222.0.33 () {54 vars in 961 bytes} [Thu Sep 22 12:32:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 12:32:09.643196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6234-9450-4d7c-9614-9edb3e91e31d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:09.643435 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6234-9450-4d7c-9614-9edb3e91e31d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:09.643708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a6234-9450-4d7c-9614-9edb3e91e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:09.663908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5e0a6234-9450-4d7c-9614-9edb3e91e31d None None] MFA Rules not processed for user `faf0bcc4ab8649cf98889340287a227c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:09.665287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5e0a6234-9450-4d7c-9614-9edb3e91e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:09.676201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244daf5-c660-4225-a8dd-885aab1a291b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:09.681285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902810-f97a-4264-b952-93b606198e18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:09.681901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902810-f97a-4264-b952-93b606198e18 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:09.682183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902810-f97a-4264-b952-93b606198e18 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:09.682457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62902810-f97a-4264-b952-93b606198e18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:09.692416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62902810-f97a-4264-b952-93b606198e18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:09.694019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62902810-f97a-4264-b952-93b606198e18 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:09.702612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1133/2270] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:32:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:32:09.735349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1138/2271] 10.222.0.33 () {58 vars in 1058 bytes} [Thu Sep 22 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:32:09.743915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902810-f97a-4264-b952-93b606198e18 service swift] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:09.758839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62902810-f97a-4264-b952-93b606198e18 service swift] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:09.772361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:09.772911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:09.773212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:09.773496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:09.776504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:09.795039 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:09.797348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:09.811747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1134/2272] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:32:19.725403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=54 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:19.805777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=38 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:28.901440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4674-49cd-45ee-9892-724ee61d6709 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:28.902122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4674-49cd-45ee-9892-724ee61d6709 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:28.902355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba4674-49cd-45ee-9892-724ee61d6709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:28.923564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9fba4674-49cd-45ee-9892-724ee61d6709 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:28.925346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9fba4674-49cd-45ee-9892-724ee61d6709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:28.933652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1139/2273] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:28.954834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:28.967564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa10cb-7c20-4464-b1d3-8456ce2a8eec service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:28.974286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:28.975134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:28.975590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:28.976005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:28.985501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:28.987501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:28.997775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1135/2274] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:32:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:32:34.951449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:34.951986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:34.951986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:34.974972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:34.976180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:34.991621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1140/2275] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:32:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:35.005273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:35.016793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b9be6-504a-41cc-a034-0560af131419 service swift] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:35.027431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:35.027431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:35.027431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:35.027833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:35.037206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:35.038892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:35.060877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1136/2276] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:32:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:32:37.491922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:37.494150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:37.509919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48bc9-8039-4614-8c69-608916a2d4e4 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:37.515032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:37.515876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:37.516405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:37.516886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] PATH_INFO: `/v3/users/70d839834f5240dd91c5f7f615ffe36f` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:37.518205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d839834f5240dd91c5f7f615ffe36f)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:37.520332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:37.572620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d839834f5240dd91c5f7f615ffe36f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.584232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d839834f5240dd91c5f7f615ffe36f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.584940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d839834f5240dd91c5f7f615ffe36f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.595668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d839834f5240dd91c5f7f615ffe36f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.608798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-262f61fa-5e39-4a37-9426-6b920655262c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d839834f5240dd91c5f7f615ffe36f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.618543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1141/2277] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:32:37 2022] DELETE /identity/v3/users/70d839834f5240dd91c5f7f615ffe36f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:32:37.825280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:37.837155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfc3d3-a039-4696-9839-6639f74a30b2 service swift] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:37.928233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:37.928914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:37.929182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:37.930816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] PATH_INFO: `/v3/projects/543e3a7b5af24948a0ad0519d000e209` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:37.932617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=543e3a7b5af24948a0ad0519d000e209)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:37.934621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:37.990104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '543e3a7b5af24948a0ad0519d000e209', 'request_id': 'req-1e9c9021-3423-494d-b2ad-0f27b9fc5983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:37.999970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '543e3a7b5af24948a0ad0519d000e209', 'request_id': 'req-1e9c9021-3423-494d-b2ad-0f27b9fc5983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:38.006917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '543e3a7b5af24948a0ad0519d000e209', 'request_id': 'req-1e9c9021-3423-494d-b2ad-0f27b9fc5983'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:38.022655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] The token cache is being invalidate because project 543e3a7b5af24948a0ad0519d000e209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:32:38.022973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:38.023859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1e9c9021-3423-494d-b2ad-0f27b9fc5983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:32:38.025544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1137/2278] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:32:37 2022] DELETE /identity/v3/projects/543e3a7b5af24948a0ad0519d000e209 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:32:38.031031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.031343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.031618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.066578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:38.067887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:38.163440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1142/2279] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:38.172406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.173359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.173709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.204442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:38.206018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:38.215462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1138/2280] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:38.226321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.231442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555f3d8-34df-4046-8a78-4479be82c6d6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.236130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.236797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.237067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.237338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.238112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:32:38.238408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.240045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.251917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1143/2281] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:32:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:32:38.256239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.267052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f443041d-c904-4112-8ae1-8a925535c027 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.272143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.272861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.273076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.273964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.274571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.276456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.319299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1139/2282] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:32:38 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:32:38.320534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.332502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07ccd1-27d9-4d0a-b0c8-e994ac664c15 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.402766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:38.422667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.423351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.423627 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.424029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.424916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.426978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.456384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1144/2283] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:32:38 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:32:38.458188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.470442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a64d0e-fbe1-423c-b6b3-10de17bb15f0 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.475155 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.476529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.476972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.477709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.478533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.480145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.619406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1140/2284] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:32:38.620760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.633608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a67ce52-d645-41b6-95a5-ad6ea13cf2c6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.638369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.638965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.639241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.639511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.640166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.642053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76ae506-0f76-4957-b302-b3ac9f962c9e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.656464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1145/2285] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:32:38.659014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.669665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35696ff-2f6e-4949-b495-b7400de7a272 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.674292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.675004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.675392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.675751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] PATH_INFO: `/v3/projects/ba0738b42f624e1b84b9677e82a31129/users/c14535e559c149f196930c367229b40a/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.690717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0738b42f624e1b84b9677e82a31129, user_id=c14535e559c149f196930c367229b40a, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.693445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.702572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:38.708401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1141/2286] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:32:38 2022] PUT /identity/v3/projects/ba0738b42f624e1b84b9677e82a31129/users/c14535e559c149f196930c367229b40a/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:32:38.718547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88225e3-d530-4ebd-a5c0-8324766e8ea2 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.718824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88225e3-d530-4ebd-a5c0-8324766e8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.719147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88225e3-d530-4ebd-a5c0-8324766e8ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.750512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d88225e3-d530-4ebd-a5c0-8324766e8ea2 None None] MFA Rules not processed for user `c14535e559c149f196930c367229b40a`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:38.751729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d88225e3-d530-4ebd-a5c0-8324766e8ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:38.832590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1146/2287] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:32:38.843629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:38.846004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:38.931410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e900a-9681-4cd3-8a9c-1b8ad998162d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:38.938343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:38.939276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:38.939709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:38.940184 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:38.954942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:38.956605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56611fe9-1dc7-4ab3-a208-a1f26f2530e1 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:38.971123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1142/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:32:38 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:32:44.673734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:44.673734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:44.674338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:44.675448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1147/2289] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:32:44.683074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ea7e5-0f37-4f3a-99be-6eca2eef9165 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:44.683970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ea7e5-0f37-4f3a-99be-6eca2eef9165 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:44.684673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ea7e5-0f37-4f3a-99be-6eca2eef9165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:44.716335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-596ea7e5-0f37-4f3a-99be-6eca2eef9165 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:44.717898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-596ea7e5-0f37-4f3a-99be-6eca2eef9165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:44.796353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1143/2290] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:44.810526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:44.812950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:44.879256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cfe019-7da8-43c1-9a37-0bee46153540 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:44.883961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:44.884595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:44.884868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:44.885134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:44.896851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:44.898526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2900a9-1f92-4829-a3d1-3c7a346f4ae6 service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:44.910258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1148/2291] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:32:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:32:44.983210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:44.983488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:44.983755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:44.985173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1144/2292] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:32:44.989934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c709-9d19-4400-9df2-a9ebc8dcb594 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:44.990220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c709-9d19-4400-9df2-a9ebc8dcb594 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:44.990493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6070c709-9d19-4400-9df2-a9ebc8dcb594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:45.010799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6070c709-9d19-4400-9df2-a9ebc8dcb594 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:45.011984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6070c709-9d19-4400-9df2-a9ebc8dcb594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:45.063324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1149/2293] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:45.075763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:45.077746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:45.091081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5439b5-d405-4008-baee-039c91873be4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:45.094548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:45.095162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:45.095435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:45.095695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:45.103312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:45.104774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08385dc0-fbd2-4f9f-a94a-02d0d5d8629c service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:45.112256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1145/2294] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:32:45.197887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dde1-232d-4ba4-a5a6-677b992c909e None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:45.198169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dde1-232d-4ba4-a5a6-677b992c909e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:45.198440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037dde1-232d-4ba4-a5a6-677b992c909e None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:45.199768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1150/2295] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:32:45.204612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:45.204891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:45.205160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:45.224684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:32:45.225849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:32:45.235099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1146/2296] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:32:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:32:45.245373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037dde1-232d-4ba4-a5a6-677b992c909e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:32:45.256340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037dde1-232d-4ba4-a5a6-677b992c909e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:32:45.260621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:32:45.261229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:32:45.261503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:32:45.261770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:32:45.268756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:45.274492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:32:45.276401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5523ac7-e699-41f8-af60-47d2e110081f service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:32:45.287036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1151/2297] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:32:55.219958 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=66 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:32:55.282452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:04.555106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:33:04.618190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dcbbcf-a785-41bc-968e-c1955f0d6096 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:33:04.623885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:33:04.624529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:04.624808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:04.625077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] PATH_INFO: `/v3/users/c14535e559c149f196930c367229b40a` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:04.626235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14535e559c149f196930c367229b40a)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:33:04.628129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:33:04.673847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14535e559c149f196930c367229b40a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:04.681885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14535e559c149f196930c367229b40a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:04.693768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14535e559c149f196930c367229b40a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:04.694389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14535e559c149f196930c367229b40a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:04.701033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14535e559c149f196930c367229b40a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:04.706658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1147/2298] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:33:04 2022] DELETE /identity/v3/users/c14535e559c149f196930c367229b40a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:33:04.711206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:04.711692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:04.711793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:04.731017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:04.732210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:04.791844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1152/2299] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:33:04.801460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:33:04.803105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:04.862501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48daeca0-3eab-4d77-942e-57654bffb6c3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:33:04.866620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:33:04.867477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:04.867621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:04.867826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:04.882477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:33:04.884107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8000ef1-b2c4-48f6-832c-a18db49e5c06 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:33:04.893595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1148/2300] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:33:05.171265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:33:05.173107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:05.186617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03c28b-a1fd-4528-a560-964b27962a25 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:33:05.191431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:33:05.192151 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:05.192523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:05.192862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] PATH_INFO: `/v3/projects/ba0738b42f624e1b84b9677e82a31129` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:05.194337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0738b42f624e1b84b9677e82a31129)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:33:05.196922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:33:05.253180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0738b42f624e1b84b9677e82a31129', 'request_id': 'req-9d716353-9317-48fa-af6d-af4b5113695c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:05.261135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0738b42f624e1b84b9677e82a31129', 'request_id': 'req-9d716353-9317-48fa-af6d-af4b5113695c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:05.266903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0738b42f624e1b84b9677e82a31129', 'request_id': 'req-9d716353-9317-48fa-af6d-af4b5113695c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:05.278874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] The token cache is being invalidate because project ba0738b42f624e1b84b9677e82a31129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:33:05.279157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:05.279806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:33:05.281321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1153/2301] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:33:05 2022] DELETE /identity/v3/projects/ba0738b42f624e1b84b9677e82a31129 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:33:09.344768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd22-80bf-45da-86aa-559ac95b947d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:09.345280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd22-80bf-45da-86aa-559ac95b947d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:09.345372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424cd22-80bf-45da-86aa-559ac95b947d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:09.372896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7424cd22-80bf-45da-86aa-559ac95b947d None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:09.374292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7424cd22-80bf-45da-86aa-559ac95b947d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:09.394884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:09.438006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1149/2302] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:33:09.462701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:33:09.464829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:09.548485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d716353-9317-48fa-af6d-af4b5113695c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:33:09.556582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:33:09.557340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:09.557598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:09.557985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:09.566991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:33:09.568751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af70fbbc-0b4d-4d9b-b418-cd916dfd2698 service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:33:09.578748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1154/2303] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:33:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:33:19.431396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=15 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:19.575742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:28.190321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debd3d-35a7-4e20-ac8f-560e65753416 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:28.190887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debd3d-35a7-4e20-ac8f-560e65753416 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:28.190887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37debd3d-35a7-4e20-ac8f-560e65753416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:28.210753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-37debd3d-35a7-4e20-ac8f-560e65753416 None None] MFA Rules not processed for user `0847f00afbde44068fcf1cedbce24be5`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:28.212486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-37debd3d-35a7-4e20-ac8f-560e65753416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:28.268469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1150/2304] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:33:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:33:38.261195 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=25 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:38.958550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984e8c-38f8-464d-8832-34655a90998d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:38.959104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984e8c-38f8-464d-8832-34655a90998d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:38.959104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28984e8c-38f8-464d-8832-34655a90998d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:38.978551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-28984e8c-38f8-464d-8832-34655a90998d None None] MFA Rules not processed for user `55a7f7ba0b27479f8d1e68d808637d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:38.979736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-28984e8c-38f8-464d-8832-34655a90998d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:39.032003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1155/2305] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:33:39.484707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6803d-9be7-48d6-bd6e-6a745aba81d2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:39.486096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6803d-9be7-48d6-bd6e-6a745aba81d2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:39.486442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6803d-9be7-48d6-bd6e-6a745aba81d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:39.512509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a1c6803d-9be7-48d6-bd6e-6a745aba81d2 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:39.513722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a1c6803d-9be7-48d6-bd6e-6a745aba81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:39.520861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1151/2306] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:33:39.534992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984e8c-38f8-464d-8832-34655a90998d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:33:39.549138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28984e8c-38f8-464d-8832-34655a90998d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:33:39.556652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:33:39.557619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:39.558117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:39.558572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:39.568256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:33:39.570277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:33:39.580749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1156/2307] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:33:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:33:39.582580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:33:39.583092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:33:39.584103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:33:39.606830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] MFA Rules not processed for user `44c2913f48ad4200b6baf14c0830770c`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:33:39.612420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:33:39.678324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1152/2308] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:33:49.576678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=31 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:33:49.670867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=28 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:10.585817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:10.587219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:10.643513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e74cb-5607-4696-b957-26255ef7979d service swift] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:10.646950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:10.647540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:10.647813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:10.648080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:10.649778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:10.717602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:10.719033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:10.730394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1157/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:13.327217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:13.328732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:13.384206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26d446-c8c6-4929-b012-db6e02bed20a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:13.388429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:13.389027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:13.389306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:13.389576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:13.391293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:13.404347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:13.405762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:13.416368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1153/2310] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:20.724955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=54 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:23.411643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=26 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:24.949954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:24.951452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:24.962026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb279b0-9195-400c-9222-a78402e6e43c service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:24.966878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:24.967221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:24.967515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:24.967787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:24.969536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:24.980965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:24.982384 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:24.991610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1158/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:34.987180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:37.993289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:38.008009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ad2bd-042c-430a-8be1-2e85dcee235a service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:38.016443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:38.017127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:38.017586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:38.017934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:38.023153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:38.094285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:38.096264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:38.112318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1154/2312] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:38.665208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:38.667746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:38.684540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea96547-6ebe-4653-accf-765c7565eaeb service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:38.693567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:38.694400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:38.694837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:38.695269 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:38.697206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:38.710763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:38.713122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:38.724711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1159/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:39.524278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:39.525921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:39.596052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:39.597630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:39.631923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db8491-4ceb-4806-8b2c-1950a91a33b2 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:39.638572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:39.639179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:39.639473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:39.640304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:39.642137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:39.657432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:39.659106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:39.668980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca57d80-1b71-47da-81f2-0bc491c74518 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:39.673836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:39.674484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:39.674754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:39.675024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] PATH_INFO: `/v3/users/44c2913f48ad4200b6baf14c0830770c` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:39.676148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44c2913f48ad4200b6baf14c0830770c)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:39.678114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:39.679013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1155/2314] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:39.732025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44c2913f48ad4200b6baf14c0830770c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:39.741959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44c2913f48ad4200b6baf14c0830770c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:39.742624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44c2913f48ad4200b6baf14c0830770c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:39.751681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44c2913f48ad4200b6baf14c0830770c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:39.765453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44c2913f48ad4200b6baf14c0830770c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:39.771997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1160/2315] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:34:39 2022] DELETE /identity/v3/users/44c2913f48ad4200b6baf14c0830770c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:40.018882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:40.029189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45ba86-de30-473d-a6a8-c030aba8f18e service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:40.089460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:40.090078 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:40.090349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:40.090613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] PATH_INFO: `/v3/projects/24db83f7bc4d4888b73d4d4ab7c4bdce` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:40.091710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24db83f7bc4d4888b73d4d4ab7c4bdce)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:40.094090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:40.137638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:40.155977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24db83f7bc4d4888b73d4d4ab7c4bdce', 'request_id': 'req-25bb2e9a-7f38-4016-96df-cb58eff5a14d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:40.163696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24db83f7bc4d4888b73d4d4ab7c4bdce', 'request_id': 'req-25bb2e9a-7f38-4016-96df-cb58eff5a14d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:40.171563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24db83f7bc4d4888b73d4d4ab7c4bdce', 'request_id': 'req-25bb2e9a-7f38-4016-96df-cb58eff5a14d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:40.185348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] The token cache is being invalidate because project 24db83f7bc4d4888b73d4d4ab7c4bdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:34:40.185920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:40.186693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-25bb2e9a-7f38-4016-96df-cb58eff5a14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:40.188732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1156/2316] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:34:40 2022] DELETE /identity/v3/projects/24db83f7bc4d4888b73d4d4ab7c4bdce => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:41.503352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:41.505083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:41.614625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:41.626991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebabb7-ef99-4f5d-a9fc-0df1985750a9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:41.732665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:41.733349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:41.734128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:41.734398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:41.738153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:41.821770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:41.823594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:41.833931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1161/2317] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:43.790637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:43.791127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:43.791468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:43.836183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] MFA Rules not processed for user `262e5327ba144a6287fef6dd6293f886`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:43.838186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:43.946660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1157/2318] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:44.273946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:44.275413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:44.331335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5733f8-06b5-4ed2-aa7e-54056b031bf2 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:44.335018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:44.335617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:44.335889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:44.336154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:44.337907 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:44.348534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:44.349968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669feb11-c371-4e5f-b29a-60a2f02b3bc2 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:44.358081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1162/2319] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:44.699288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:44.705503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:44.806072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0908cd7-2026-4d8e-90ec-413d69b2e826 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:44.814479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:44.815206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:44.815573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:44.815910 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:44.817991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:44.831262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:44.832915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:44.844680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1158/2320] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:46.513895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:46.514450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:46.514545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:46.534064 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] MFA Rules not processed for user `49c772b0c2bb49a4ab4048d980bd20a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:46.535259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:46.578012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:46.593227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1163/2321] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:46.892789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:46.901977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b451d7-0277-459f-9d1b-a811ef92db64 service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:46.905861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:46.906434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:46.906703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:46.906963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:46.908656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:46.917656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:46.919053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:46.927249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1159/2322] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:49.971880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:49.987311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f73ab3-ecd7-451a-a3ba-dc69af9c62cc None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:49.992510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:49.993451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:49.993996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:49.994473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:50.014992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:50.027421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:50.029337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:50.042961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1164/2323] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:51.094779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:51.096199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:51.106593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749dbb1-2abc-4f37-bb03-90fe785e1bbb service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:51.110142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:51.110725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:51.110999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:51.111263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:51.112919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:51.156429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:51.183392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:51.185253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:51.196149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1160/2324] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:34:54.735150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:54.747258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555cbb1-3fcb-41f6-a604-7a580632770f service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:54.753052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:54.753652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:54.753925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:54.754192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:54.755920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:54.767557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:54.769391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:54.779784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1165/2325] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:54.816503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:54.818172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:54.831458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768542-53c2-4530-ba52-d7be25850d73 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:54.835598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:54.836283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:54.836648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:54.837005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] PATH_INFO: `/v3/users/0847f00afbde44068fcf1cedbce24be5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:54.838165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0847f00afbde44068fcf1cedbce24be5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:54.840125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:54.890811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0847f00afbde44068fcf1cedbce24be5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:54.898531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0847f00afbde44068fcf1cedbce24be5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:54.911699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0847f00afbde44068fcf1cedbce24be5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:54.912391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0847f00afbde44068fcf1cedbce24be5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:54.919033 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f999a86-beeb-4b1c-bd6c-8fd2b06fb22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0847f00afbde44068fcf1cedbce24be5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:54.925350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1161/2326] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:34:54 2022] DELETE /identity/v3/users/0847f00afbde44068fcf1cedbce24be5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:55.165028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.180555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ca8b4-175b-40ee-86c5-035ba5887d57 service placement] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.247614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.248470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.249300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.249651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] PATH_INFO: `/v3/projects/fb62ca26c4fe47deb0a21511a96e6403` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.250999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb62ca26c4fe47deb0a21511a96e6403)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.253150 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.298104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb62ca26c4fe47deb0a21511a96e6403', 'request_id': 'req-e19d7917-608f-48cf-9fa9-92dae321b765'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:55.304871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb62ca26c4fe47deb0a21511a96e6403', 'request_id': 'req-e19d7917-608f-48cf-9fa9-92dae321b765'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:55.310355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb62ca26c4fe47deb0a21511a96e6403', 'request_id': 'req-e19d7917-608f-48cf-9fa9-92dae321b765'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:55.321515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] The token cache is being invalidate because project fb62ca26c4fe47deb0a21511a96e6403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:34:55.321803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:55.322422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-e19d7917-608f-48cf-9fa9-92dae321b765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:55.323876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1166/2327] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:34:55 2022] DELETE /identity/v3/projects/fb62ca26c4fe47deb0a21511a96e6403 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:55.329383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.329663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.329931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.348700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:55.349885 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:55.407555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1162/2328] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:55.415078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.415356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.415624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.434875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:55.436634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:55.444377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1167/2329] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:55.448459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.458445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c542e-3a90-474f-8131-bf5a1a2b8053 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.462902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.463486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.463761 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.464029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.464684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:34:55.464982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.466548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.478053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1163/2330] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:34:55.482111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.491616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cedc51c-2319-49a6-a6d4-f9fa56f445cc None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.496205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.496846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.497123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.497387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.498010 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.500508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.526347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1168/2331] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:34:55 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:34:55.530301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.540333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304b98-d6c9-462f-b96c-99063888b092 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.572561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:55.605189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.605795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.606073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.606375 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.607014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.609588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.634411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1164/2332] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:34:55 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:34:55.638578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.653082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660be3-e145-4e02-8fc0-b5bd8d40f64b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.658440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.659050 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.659328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.659609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.660240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.661884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.673950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1169/2333] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:34:55.678045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.689154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11066a3a-cff2-4663-af7e-af294fe36648 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.697015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.697703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.698042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.698316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.698953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.700566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3653c90-2d3b-455e-a29c-c512b913f7ce admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.717519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1165/2334] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:34:55.721557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.738920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764d779-7f91-4c10-96ef-336c9de4c602 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.744029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.744746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.745021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.745292 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] PATH_INFO: `/v3/projects/77edb8cdc8294a72aaa2c3dbc5fe88dd/users/5801b6cdf61c4264b3a7839dfa94064a/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.759289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:55.763354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77edb8cdc8294a72aaa2c3dbc5fe88dd, user_id=5801b6cdf61c4264b3a7839dfa94064a, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.766684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.780473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1170/2335] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:34:55 2022] PUT /identity/v3/projects/77edb8cdc8294a72aaa2c3dbc5fe88dd/users/5801b6cdf61c4264b3a7839dfa94064a/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:55.791540 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.791794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.792077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.822654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] MFA Rules not processed for user `5801b6cdf61c4264b3a7839dfa94064a`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:55.823860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:55.887139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1166/2336] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:55.897742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:55.899441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:55.971761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65648-c29d-4628-9b31-b5aa2736bed4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:55.977236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:55.978104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:55.978372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:55.978642 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:55.987192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:55.988878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2da7-4b6e-4728-a2f9-355555ec6563 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:55.997960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1171/2337] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:56.218902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:56.297451 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce511f29-d9e6-457d-8a49-fe47765f459f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:56.301959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:56.302654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:56.303001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:56.303341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] PATH_INFO: `/v3/users/5801b6cdf61c4264b3a7839dfa94064a` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:56.304586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5801b6cdf61c4264b3a7839dfa94064a)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:56.306589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:56.369290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:56.371933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5801b6cdf61c4264b3a7839dfa94064a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:56.381216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5801b6cdf61c4264b3a7839dfa94064a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:56.395050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5801b6cdf61c4264b3a7839dfa94064a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:56.395720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5801b6cdf61c4264b3a7839dfa94064a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:56.402780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5801b6cdf61c4264b3a7839dfa94064a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:56.409295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1167/2338] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:34:56 2022] DELETE /identity/v3/users/5801b6cdf61c4264b3a7839dfa94064a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:56.413841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:56.414126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:56.414399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:56.433951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:56.435130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:56.493502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1172/2339] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:56.503888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:56.505335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:56.563105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f315050-f655-4fa3-bb6e-c5ebc064da65 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:56.566725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:56.567325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:56.567602 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:56.567874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:56.576866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:56.578294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f7cb9-a5d4-4a9d-9ec6-dbb3756b3727 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:56.587832 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1168/2340] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:57.187163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.188713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:57.202115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9897bf4-18be-4c2b-bb68-04bd27e24c46 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.206770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.207408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.207684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.207952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] PATH_INFO: `/v3/projects/77edb8cdc8294a72aaa2c3dbc5fe88dd` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.209136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=77edb8cdc8294a72aaa2c3dbc5fe88dd)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.211044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.260897 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77edb8cdc8294a72aaa2c3dbc5fe88dd', 'request_id': 'req-3f27e530-ae38-46fa-9743-8cbdb63125ba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:57.270772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77edb8cdc8294a72aaa2c3dbc5fe88dd', 'request_id': 'req-3f27e530-ae38-46fa-9743-8cbdb63125ba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:57.277322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77edb8cdc8294a72aaa2c3dbc5fe88dd', 'request_id': 'req-3f27e530-ae38-46fa-9743-8cbdb63125ba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:57.285834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:57.291334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] The token cache is being invalidate because project 77edb8cdc8294a72aaa2c3dbc5fe88dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:34:57.291637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:57.292594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3f27e530-ae38-46fa-9743-8cbdb63125ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:34:57.299454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1173/2341] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:34:57 2022] DELETE /identity/v3/projects/77edb8cdc8294a72aaa2c3dbc5fe88dd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:57.301277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.301768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.301894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.329603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:57.331092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:57.401339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1169/2342] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:34:57.403270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.403553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.403827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.429177 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:57.430637 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:57.440684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1174/2343] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:57.445392 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.456455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12151a3-41e7-4bdb-a71b-435d29038b8b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.462785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.463041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.463324 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.463603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.464249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:34:57.464749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.466339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.487271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.500393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1170/2344] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:34:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:34:57.500523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ab2f4-71fd-4f80-90e9-6d2593f5bf51 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.505647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.506323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.506669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.507018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.507720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.509845 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.538055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1175/2345] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:34:57.542225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.553088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498870f0-ec62-47f4-b62e-3c9463074d09 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.615508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:57.633916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.635113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.635463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.635838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.637560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.640753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.664446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1171/2346] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/users => generated 301 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:34:57.669540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.690515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cccd79c-1883-4206-8f1b-ef564e06d6fb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.698708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.700180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.700655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.700987 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.703720 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.705501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.718275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1176/2347] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:34:57.722329 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.733606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908aca34-a019-4038-a2d6-7bf55677dcc1 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.740262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.741036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.741382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.741726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.742444 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.744179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008c5cd5-c1be-4c25-851f-51ba5b2e8563 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.756961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1172/2348] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:34:57.761169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.773921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9aa8fb-8e33-4e2c-b6db-d5424e2a02d5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:57.779370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:57.780100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.780799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.780799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] PATH_INFO: `/v3/projects/49ca1a3ff09d4a2fb8c5ab97435b1ffe/users/b9d88b6cba2e42419fe90f493d62ac0e/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.798292 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49ca1a3ff09d4a2fb8c5ab97435b1ffe, user_id=b9d88b6cba2e42419fe90f493d62ac0e, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:57.800972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:57.827060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1177/2349] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:34:57 2022] PUT /identity/v3/projects/49ca1a3ff09d4a2fb8c5ab97435b1ffe/users/b9d88b6cba2e42419fe90f493d62ac0e/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:57.829398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:57.830012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:57.830401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:57.870680 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] MFA Rules not processed for user `b9d88b6cba2e42419fe90f493d62ac0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:57.872139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:57.918331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:57.920102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:57.986661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1173/2350] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:57.990954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.033449 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60234c-63dc-411c-b022-878c09fc5348 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.046325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.046891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.048078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.048408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.051785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:58.071365 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10256d1-8491-4cd9-878d-a9ce948096c2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.071509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.072949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.076516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.077274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.077963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.077963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.078581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.080513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.086826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1178/2351] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:34:58.115808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1174/2352] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:34:57 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:34:58.119579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.132235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242b272-2ca3-4beb-bdb2-774dd5114560 service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.213229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.213915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.214121 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.214584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.215385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.218130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.257812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1179/2353] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:34:58 2022] POST /identity/v3/users => generated 306 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:34:58.259792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.280420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d91927-f460-49f7-9cb9-c95594894541 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.289051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.289701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.290591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.290867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.291526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.293391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.324239 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1175/2354] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:34:58.328128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.345175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b38758-2641-4272-bba8-f8f1b36b0112 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.351253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.352045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.352437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.353769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] PATH_INFO: `/v3/projects/cbfd5e214f50465396a251a3cefb90d3/users/dfa705d4c4834ed793f01674972cd299/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.358567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:58.378021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfd5e214f50465396a251a3cefb90d3, user_id=dfa705d4c4834ed793f01674972cd299, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.381524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.399240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1180/2355] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:34:58 2022] PUT /identity/v3/projects/cbfd5e214f50465396a251a3cefb90d3/users/dfa705d4c4834ed793f01674972cd299/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:34:58.404490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.417369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b7ceb-8802-4c11-a7ce-599eb4d7478e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.453704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:34:58.485438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.486228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.486503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.486898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.487531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.489156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28bea4-4937-4a69-94d3-9f5d913d61d5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.499006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1176/2356] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:34:58.503718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.515006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817028a1-3fd3-4e4f-a17e-92913c950edc admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.532072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.533732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.533732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.533732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] PATH_INFO: `/v3/projects/cbfd5e214f50465396a251a3cefb90d3/users/dfa705d4c4834ed793f01674972cd299/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.535338 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbfd5e214f50465396a251a3cefb90d3, user_id=dfa705d4c4834ed793f01674972cd299, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.538022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.549132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1181/2357] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:34:58 2022] PUT /identity/v3/projects/cbfd5e214f50465396a251a3cefb90d3/users/dfa705d4c4834ed793f01674972cd299/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:34:58.560927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.561222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.561494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.589716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] MFA Rules not processed for user `dfa705d4c4834ed793f01674972cd299`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:34:58.591291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:58.667282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1177/2358] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:34:58.692808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.695754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:34:58.792819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e6fe-65e3-4f79-bfd3-b8af671b4a71 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.798174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.798923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.799201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.799470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:58.813988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:58.815437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d2607-bcb7-4677-88c0-072c29849341 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:58.826489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1182/2359] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:34:58.925784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:34:58.938216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c2269-0459-4175-8ba2-b2e532039587 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:34:58.942764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:34:58.944169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:34:58.944570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:34:58.944917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:34:59.018221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:34:59.021690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:34:59.036479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1178/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:02.835516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982e0e-9283-4ed7-89e6-c34cb80a88a5 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:02.836035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982e0e-9283-4ed7-89e6-c34cb80a88a5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:02.836035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51982e0e-9283-4ed7-89e6-c34cb80a88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:02.863331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-51982e0e-9283-4ed7-89e6-c34cb80a88a5 None None] MFA Rules not processed for user `55a7f7ba0b27479f8d1e68d808637d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:02.863331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-51982e0e-9283-4ed7-89e6-c34cb80a88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:02.925918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1183/2361] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:03.323698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:03.386208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:03.404843 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c629c-3c4c-4538-82c3-e017fd915055 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:03.410389 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:03.411827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:03.411827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:03.411827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] PATH_INFO: `/v3/users/b9d88b6cba2e42419fe90f493d62ac0e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:03.412906 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d88b6cba2e42419fe90f493d62ac0e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:03.414834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:03.468888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d88b6cba2e42419fe90f493d62ac0e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:03.479561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d88b6cba2e42419fe90f493d62ac0e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:03.494019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d88b6cba2e42419fe90f493d62ac0e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:03.494896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d88b6cba2e42419fe90f493d62ac0e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:03.504357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d88b6cba2e42419fe90f493d62ac0e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:03.511659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1179/2362] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:03 2022] DELETE /identity/v3/users/b9d88b6cba2e42419fe90f493d62ac0e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:03.516480 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:03.516766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:03.517142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:03.536815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:03.538080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:03.607094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1184/2363] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:03.623966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:03.625599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:03.722231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aabeaf6-a072-4ca6-ab89-1504bb75e6ff admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:03.726807 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:03.727410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:03.727969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:03.728221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:03.738628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:03.739915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:03.755633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1180/2364] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:04.045614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:04.047162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:04.072592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597c7358-c46d-4ee7-9a48-627d5d2eadd5 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:04.080695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:04.081457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:04.082396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:04.082900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] PATH_INFO: `/v3/projects/49ca1a3ff09d4a2fb8c5ab97435b1ffe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:04.084218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] RBAC: Authorizing `identity:delete_project(project_id=49ca1a3ff09d4a2fb8c5ab97435b1ffe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:04.087225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:04.124481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:04.125554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:04.126575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:04.173753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49ca1a3ff09d4a2fb8c5ab97435b1ffe', 'request_id': 'req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.183397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49ca1a3ff09d4a2fb8c5ab97435b1ffe', 'request_id': 'req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.192658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49ca1a3ff09d4a2fb8c5ab97435b1ffe', 'request_id': 'req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.209416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] The token cache is being invalidate because project 49ca1a3ff09d4a2fb8c5ab97435b1ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:04.209844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.212640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.214469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1185/2365] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:04 2022] DELETE /identity/v3/projects/49ca1a3ff09d4a2fb8c5ab97435b1ffe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:04.219147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:04.220839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:04.230331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628951e4-4d62-4814-a5c5-795af35c2b50 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:04.243768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:04.310353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dfe33-5a47-44ee-9ab4-aa85fcaf832e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:04.310878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:04.311567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:04.311976 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:04.312286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:04.314349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:04.317738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:04.318573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:04.318934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:04.319283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] PATH_INFO: `/v3/users/dfa705d4c4834ed793f01674972cd299` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:04.321451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa705d4c4834ed793f01674972cd299)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:04.323668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:04.328274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:04.329905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:04.341696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1181/2366] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:35:04.383548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa705d4c4834ed793f01674972cd299'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.397081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa705d4c4834ed793f01674972cd299'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.398048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa705d4c4834ed793f01674972cd299'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.407537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa705d4c4834ed793f01674972cd299'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.429600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6deca8c0-12d8-48c5-9c03-34185585972d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa705d4c4834ed793f01674972cd299'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:04.447623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1186/2367] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:04 2022] DELETE /identity/v3/users/dfa705d4c4834ed793f01674972cd299 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:04.767121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:04.780890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211e48c-d8d2-4afd-9ec9-fb6a48a31921 service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:04.910022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:04.910756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:04.912463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:04.914248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] PATH_INFO: `/v3/projects/cbfd5e214f50465396a251a3cefb90d3` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:04.917206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbfd5e214f50465396a251a3cefb90d3)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:04.920031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:04.991207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbfd5e214f50465396a251a3cefb90d3', 'request_id': 'req-da619aa0-68ae-4aef-849b-dcda653f9a55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:05.002464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbfd5e214f50465396a251a3cefb90d3', 'request_id': 'req-da619aa0-68ae-4aef-849b-dcda653f9a55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:05.013123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbfd5e214f50465396a251a3cefb90d3', 'request_id': 'req-da619aa0-68ae-4aef-849b-dcda653f9a55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:05.028690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] The token cache is being invalidate because project cbfd5e214f50465396a251a3cefb90d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:05.029077 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:05.029870 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da619aa0-68ae-4aef-849b-dcda653f9a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:05.037011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.037143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.038953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.040837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1182/2368] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:04 2022] DELETE /identity/v3/projects/cbfd5e214f50465396a251a3cefb90d3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:05.064468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:05.067165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:05.092517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:05.161973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1187/2369] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:05.169862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.170178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.170452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.196657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:05.198093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:05.213665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.214583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1183/2370] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:05.223807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d03ddcd-a1cc-4bb8-be03-53b0fb7d316c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.229093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.229708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.229986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.230259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.230906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:35:05.231212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.233420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.246240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1188/2371] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:35:05.251215 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.260740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1817a91-bf60-47fa-b9ac-34c49b8752ec None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.266400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.267008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.267285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.267552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.277092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.279151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.303707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:05.313628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1184/2372] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:35:05.318245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.330057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c2186-fcdf-40a3-bcad-f76355bceed6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.402648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.403275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.403750 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.403865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.404513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.406564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.433398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1189/2373] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:05.437646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.449081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41e030-2767-4223-bafd-927e34f673ad admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.454259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.454875 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.455216 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.455494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.456172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.457809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.471182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1185/2374] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:05.472367 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.482547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ba1e-7c80-4fe0-8ba0-7dea4fd820b9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.490224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.490826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.491098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.491363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.491985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.493615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657d37e4-d759-40b5-9e63-f91672063558 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.504740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1190/2375] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:05.508920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.518972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4101af8f-7874-4735-9517-35866b43cf75 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.523583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.524342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.524891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.525318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] PATH_INFO: `/v3/projects/2bde8759a8bb46cb91bd7def0c9619b3/users/e5aef277215a4c4ba55908a0fd40f2a5/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.540675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bde8759a8bb46cb91bd7def0c9619b3, user_id=e5aef277215a4c4ba55908a0fd40f2a5, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.543665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.556814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1186/2376] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:05 2022] PUT /identity/v3/projects/2bde8759a8bb46cb91bd7def0c9619b3/users/e5aef277215a4c4ba55908a0fd40f2a5/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:35:05.567673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.567966 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.568217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.596079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] MFA Rules not processed for user `e5aef277215a4c4ba55908a0fd40f2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:05.597318 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:05.667892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1191/2377] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:05.673188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.735428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54574eb-1b83-46f1-8c90-5497caea9ba0 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.741594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.742194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.742465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.742734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.743388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.745299 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.772099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1187/2378] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:05.776152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.787824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ff3e5-a195-4469-9dfc-6e1aeb2a1654 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.806164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:05.862995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.863773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.864057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.864359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.865023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.866942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.899711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.900399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1192/2379] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:05 2022] POST /identity/v3/users => generated 299 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:05.915932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95655d5e-3296-4897-b07f-0d8570288e9a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.923164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.924001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.924484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.925231 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.926584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.928227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:05.941632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1188/2380] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:05.945686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:05.957303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a743a4-59a3-4968-af97-b83542e2b7ec admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:05.962521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:05.963260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:05.963605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:05.964060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] PATH_INFO: `/v3/projects/38050dde97e44015add1c72446a368ad/users/f797ec1191144ba2b35465d7ab1b4a1e/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:05.982954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38050dde97e44015add1c72446a368ad, user_id=f797ec1191144ba2b35465d7ab1b4a1e, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:05.986032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.001199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1193/2381] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:05 2022] PUT /identity/v3/projects/38050dde97e44015add1c72446a368ad/users/f797ec1191144ba2b35465d7ab1b4a1e/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:06.005274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:06.017233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70f172-04c6-44a5-84bd-2e5f050d21f0 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:06.081975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:06.082627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.082774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.083026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.083715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:06.085291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cf8b1b-4dc7-4a64-95e6-5aa47549de4f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.096131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1189/2382] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:06.100525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:06.112904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4fab12-5d79-457d-827f-3f58e1836bf0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:06.118623 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:06.119458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.119733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.119997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] PATH_INFO: `/v3/projects/38050dde97e44015add1c72446a368ad/users/f797ec1191144ba2b35465d7ab1b4a1e/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.121962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=38050dde97e44015add1c72446a368ad, user_id=f797ec1191144ba2b35465d7ab1b4a1e, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:06.124719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.142423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1194/2383] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:06 2022] PUT /identity/v3/projects/38050dde97e44015add1c72446a368ad/users/f797ec1191144ba2b35465d7ab1b4a1e/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:35:06.153913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.154197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.154554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.183564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] MFA Rules not processed for user `f797ec1191144ba2b35465d7ab1b4a1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:06.184847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:06.209131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:06.255636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1190/2384] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:06.266714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:06.268539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:06.329916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5a2ba-9b5c-4232-9a56-b2c1b11e99cb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:06.335778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:06.336530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.336810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.337081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.346979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:06.348634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247914d7-87f2-4aeb-9157-ef734445f8a6 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.359215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1195/2385] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:06.501874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:06.564664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612eab01-4992-43ce-8337-c9d78656dde6 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:06.569206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:06.569837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.570114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.570524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] PATH_INFO: `/v3/users/e5aef277215a4c4ba55908a0fd40f2a5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.571636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5aef277215a4c4ba55908a0fd40f2a5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:06.573583 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.630117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5aef277215a4c4ba55908a0fd40f2a5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:06.643158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5aef277215a4c4ba55908a0fd40f2a5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:06.665706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5aef277215a4c4ba55908a0fd40f2a5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:06.666671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5aef277215a4c4ba55908a0fd40f2a5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:06.675716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5aef277215a4c4ba55908a0fd40f2a5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:06.683985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1191/2386] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:06 2022] DELETE /identity/v3/users/e5aef277215a4c4ba55908a0fd40f2a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:35:06.689508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.689868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.690386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.712519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:06.714446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:06.774807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1196/2387] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:06.784845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:06.786273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:06.830708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:06.848738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d8cf0-95b5-4e27-be3a-7eeb83f1ec3d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:06.852448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:06.853361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:06.853774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:06.854046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:06.864588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:06.866864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:06.879565 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1192/2388] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:07.145692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:07.147162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:07.159699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db72a5-b2c4-4b6b-9547-1a96574f36d2 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:07.175858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:07.176858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:07.177142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:07.178160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] PATH_INFO: `/v3/projects/2bde8759a8bb46cb91bd7def0c9619b3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:07.179539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bde8759a8bb46cb91bd7def0c9619b3)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:07.181914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:07.191137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:07.236578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bde8759a8bb46cb91bd7def0c9619b3', 'request_id': 'req-cbb553fe-350f-4590-bc0e-8013e64d9bb8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.245259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bde8759a8bb46cb91bd7def0c9619b3', 'request_id': 'req-cbb553fe-350f-4590-bc0e-8013e64d9bb8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.251640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bde8759a8bb46cb91bd7def0c9619b3', 'request_id': 'req-cbb553fe-350f-4590-bc0e-8013e64d9bb8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.262982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] The token cache is being invalidate because project 2bde8759a8bb46cb91bd7def0c9619b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:07.263268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.263961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.265459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1197/2389] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:07 2022] DELETE /identity/v3/projects/2bde8759a8bb46cb91bd7def0c9619b3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:07.269499 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:07.278767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177d491-d656-4ac0-b24c-8e958e6cabdf service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:07.343277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:07.343960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:07.344251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:07.344678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] PATH_INFO: `/v3/users/f797ec1191144ba2b35465d7ab1b4a1e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:07.345787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f797ec1191144ba2b35465d7ab1b4a1e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:07.347718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:07.392644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f797ec1191144ba2b35465d7ab1b4a1e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.399345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f797ec1191144ba2b35465d7ab1b4a1e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.411394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f797ec1191144ba2b35465d7ab1b4a1e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.412027 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f797ec1191144ba2b35465d7ab1b4a1e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.419157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-00e1570a-4cc8-4a9e-b7f4-581d1cd4718d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f797ec1191144ba2b35465d7ab1b4a1e'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.427290 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1193/2390] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:07 2022] DELETE /identity/v3/users/f797ec1191144ba2b35465d7ab1b4a1e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:07.701062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:07.703382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:07.717500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb553fe-350f-4590-bc0e-8013e64d9bb8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:07.774011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:07.774568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:07.774846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:07.775117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] PATH_INFO: `/v3/projects/38050dde97e44015add1c72446a368ad` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:07.776214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38050dde97e44015add1c72446a368ad)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:07.778139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:07.823193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38050dde97e44015add1c72446a368ad', 'request_id': 'req-657089b3-26ed-478b-ae05-9841eea4e530'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.830501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38050dde97e44015add1c72446a368ad', 'request_id': 'req-657089b3-26ed-478b-ae05-9841eea4e530'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.836023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38050dde97e44015add1c72446a368ad', 'request_id': 'req-657089b3-26ed-478b-ae05-9841eea4e530'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.846813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] The token cache is being invalidate because project 38050dde97e44015add1c72446a368ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:07.847102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.847767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-657089b3-26ed-478b-ae05-9841eea4e530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:07.849307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1198/2391] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:07 2022] DELETE /identity/v3/projects/38050dde97e44015add1c72446a368ad => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:07.854593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:07.855083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:07.855238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:07.878813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:07.880021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:07.941006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1194/2392] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:07.948662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:07.948942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:07.949218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:07.970540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:07.971781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:07.979961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1199/2393] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:07.984415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:07.989212 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:07.994754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0fcd47-8439-4ab1-a754-7fafbbd2878e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.000917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.001792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.002196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.002699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.003350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:35:08.003683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.005354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.018889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1195/2394] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:35:08.023283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.032629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a309d3-e7fd-454d-a78c-dd534bba1a3c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.037289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.037884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.038164 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.038440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.039077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.041048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.062081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:08.074043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1200/2395] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:08.075151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.086334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc7867-f824-42fe-bc2f-de48358fbc15 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.144578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.145178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.145455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.145723 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.146348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.148241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.172355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1196/2396] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:08.174806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.185043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a804de-7c24-4894-b31b-00ffc5889c7a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.190227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.190826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.191103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.191372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.192008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.193632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.204459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1201/2397] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:08.208662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.218482 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f1321-ef6b-4a66-a303-4387bd462eee admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.223492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.224092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.224408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.224678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.225316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.227010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf9a000-6ed3-40b0-bdd4-21682d85e403 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.237978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1197/2398] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:08.242227 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.252898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd741d-cda8-474d-9e13-7a4718c5dbb9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.263179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.263894 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.264178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.264490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] PATH_INFO: `/v3/projects/532e594b36804a21b57a4e14b0243753/users/66eb9dcc1ecd4abf8304fb738c1c63d6/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.279257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532e594b36804a21b57a4e14b0243753, user_id=66eb9dcc1ecd4abf8304fb738c1c63d6, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.281964 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.297007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1202/2399] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:08 2022] PUT /identity/v3/projects/532e594b36804a21b57a4e14b0243753/users/66eb9dcc1ecd4abf8304fb738c1c63d6/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:08.308064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.308471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.308813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.338721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] MFA Rules not processed for user `66eb9dcc1ecd4abf8304fb738c1c63d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:08.339931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:08.409543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1198/2400] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:08.414006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.476545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd44bd8-fe99-4607-84e6-b97eb8dadb41 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.481527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.482128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.482406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.483220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.483847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.485758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.513382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1203/2401] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:08.517595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.528020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe557ab-86a7-42fe-a408-e6b477c957e9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.577129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:08.584091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.584895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.585406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.585679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.586308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.588210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.610185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1199/2402] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:08.614225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.624852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6744f-70d8-4ebe-8cf4-2b56bf03c844 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.629592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.630193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.630497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.630755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.631499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.633279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.644667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1204/2403] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:08.648834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.659441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b4fa4-8018-4302-8c08-d624b65cf563 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.663938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.664560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.664836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.665105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.665732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.667297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b8bdca-885d-44dd-8c55-b5bb220bcd74 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.677509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1200/2404] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:08.681522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.692605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140e0b8-d91a-4cd0-8572-035cc228ea6a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.697239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.697893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.698149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.698419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] PATH_INFO: `/v3/projects/0285d8367ffa49fe8aa1fed079be7eb0/users/1f37bd08c9ce41e598ed82fa8e62df65/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.712517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0285d8367ffa49fe8aa1fed079be7eb0, user_id=1f37bd08c9ce41e598ed82fa8e62df65, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.715130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.728231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1205/2405] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:08 2022] PUT /identity/v3/projects/0285d8367ffa49fe8aa1fed079be7eb0/users/1f37bd08c9ce41e598ed82fa8e62df65/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:08.739134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.739411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.739679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.766334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] MFA Rules not processed for user `1f37bd08c9ce41e598ed82fa8e62df65`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:08.767516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:08.831684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1201/2406] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:08.836215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.897507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdc6be6-6e4f-40d3-939e-953de9308f76 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:08.902059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:08.902649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:08.902921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:08.903187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:08.903793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:08.906296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:08.926577 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:08.934889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1206/2407] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:08.938929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:08.949502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009d6e8-d3dd-4919-b8ab-f5f71db52bba None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.005207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.005803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.006083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.006357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.006993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.008963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.031712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1202/2408] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:08 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:35:09.035732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.046425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad55ab4-2475-45ca-bf6a-d798d8519103 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.051397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.052069 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.052365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.052690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.053416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.055094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.066062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1207/2409] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:09.070381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.081015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb12e24-0dd2-464c-ae99-33870084d40f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.085585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.086166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.086442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.086711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] PATH_INFO: `/v3/projects/7a1996c50eb5483f9cb85a761bed2aec/users/eb2e0f8627fc4603a10b5ec0b9a1642b/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.102863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1996c50eb5483f9cb85a761bed2aec, user_id=eb2e0f8627fc4603a10b5ec0b9a1642b, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.105545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.119149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1203/2410] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:09 2022] PUT /identity/v3/projects/7a1996c50eb5483f9cb85a761bed2aec/users/eb2e0f8627fc4603a10b5ec0b9a1642b/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:09.123278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.133521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f29603-1aa2-427a-b0d3-d06c541aeab2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.190088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.190736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.190949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.191219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.191846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.193483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11f76e-bc40-4c91-b575-25c033666d11 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.203545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1208/2411] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:09.207656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.218553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd64ede1-9d70-4e29-9337-33bf2700caaa admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.223183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.223821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.224100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.224415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] PATH_INFO: `/v3/projects/7a1996c50eb5483f9cb85a761bed2aec/users/eb2e0f8627fc4603a10b5ec0b9a1642b/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.226284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1996c50eb5483f9cb85a761bed2aec, user_id=eb2e0f8627fc4603a10b5ec0b9a1642b, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.229189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.239140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1204/2412] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:09 2022] PUT /identity/v3/projects/7a1996c50eb5483f9cb85a761bed2aec/users/eb2e0f8627fc4603a10b5ec0b9a1642b/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:09.249573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.249856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.250131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.277259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] MFA Rules not processed for user `eb2e0f8627fc4603a10b5ec0b9a1642b`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:09.278447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:09.345912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1209/2413] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:09.358046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.359767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:09.380520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:09.417656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07630718-13fd-45ca-9fea-edf1971c9b7d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.421238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.421830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.422107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.422377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.431096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.432544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.442234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1205/2414] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:09.483251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:09.484699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:09.496130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c0ec4-94bf-4307-8172-97acfa118d5c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:09.500366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:09.500949 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:09.501218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:09.501482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:09.556861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:09.558293 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d78e07-0422-421c-8fff-59eb692ee7ce service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:09.567959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1210/2415] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:12.462103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:12.522997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7dd1-56cd-45b1-9044-06b3422ac193 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:12.527426 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:12.528035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:12.528332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:12.528608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] PATH_INFO: `/v3/users/66eb9dcc1ecd4abf8304fb738c1c63d6` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:12.529691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66eb9dcc1ecd4abf8304fb738c1c63d6)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:12.531595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:12.575327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66eb9dcc1ecd4abf8304fb738c1c63d6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:12.582379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66eb9dcc1ecd4abf8304fb738c1c63d6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:12.594608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66eb9dcc1ecd4abf8304fb738c1c63d6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:12.595230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66eb9dcc1ecd4abf8304fb738c1c63d6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:12.601955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66eb9dcc1ecd4abf8304fb738c1c63d6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:12.607890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1206/2416] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:12 2022] DELETE /identity/v3/users/66eb9dcc1ecd4abf8304fb738c1c63d6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:12.613021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:12.613487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:12.613600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:12.626612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:12.633093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:12.634193 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:12.695192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1211/2417] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:12.706522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:12.709945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:12.755529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:12.770656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54693aff-b1df-4c6e-8c4d-4c67de3b7675 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:12.774213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:12.774812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:12.775083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:12.775349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:12.785085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:12.786546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:12.795782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1207/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:13.070580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:13.072094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:13.085534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6ac58-13c3-4251-b56a-b700ade3a46f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:13.090504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:13.091258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:13.091624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:13.091958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] PATH_INFO: `/v3/projects/532e594b36804a21b57a4e14b0243753` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:13.093901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=532e594b36804a21b57a4e14b0243753)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:13.095863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:13.146030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532e594b36804a21b57a4e14b0243753', 'request_id': 'req-487b22ea-e179-4725-8e99-dfd1c0cad8da'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.153490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532e594b36804a21b57a4e14b0243753', 'request_id': 'req-487b22ea-e179-4725-8e99-dfd1c0cad8da'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.159849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532e594b36804a21b57a4e14b0243753', 'request_id': 'req-487b22ea-e179-4725-8e99-dfd1c0cad8da'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.175870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] The token cache is being invalidate because project 532e594b36804a21b57a4e14b0243753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:13.176156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.176952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.178450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1212/2419] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:13 2022] DELETE /identity/v3/projects/532e594b36804a21b57a4e14b0243753 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:13.182557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:13.192354 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60756bb-7955-4805-9f1b-8daf52615dd5 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:13.249894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:13.250516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:13.250798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:13.251071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] PATH_INFO: `/v3/users/1f37bd08c9ce41e598ed82fa8e62df65` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:13.252160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f37bd08c9ce41e598ed82fa8e62df65)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:13.254104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:13.299092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f37bd08c9ce41e598ed82fa8e62df65'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.306535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f37bd08c9ce41e598ed82fa8e62df65'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.322670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f37bd08c9ce41e598ed82fa8e62df65'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.323374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f37bd08c9ce41e598ed82fa8e62df65'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.331495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f37bd08c9ce41e598ed82fa8e62df65'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.337480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1208/2420] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:13 2022] DELETE /identity/v3/users/1f37bd08c9ce41e598ed82fa8e62df65 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:35:13.599997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:13.601580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:13.615517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b22ea-e179-4725-8e99-dfd1c0cad8da admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:13.686602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:13.687160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:13.687430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:13.687700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] PATH_INFO: `/v3/projects/0285d8367ffa49fe8aa1fed079be7eb0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:13.688974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0285d8367ffa49fe8aa1fed079be7eb0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:13.691004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:13.726454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:13.739823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0285d8367ffa49fe8aa1fed079be7eb0', 'request_id': 'req-b39e5dd4-00e3-4cec-9323-2384435b849a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.746382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0285d8367ffa49fe8aa1fed079be7eb0', 'request_id': 'req-b39e5dd4-00e3-4cec-9323-2384435b849a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.751951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0285d8367ffa49fe8aa1fed079be7eb0', 'request_id': 'req-b39e5dd4-00e3-4cec-9323-2384435b849a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.763033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] The token cache is being invalidate because project 0285d8367ffa49fe8aa1fed079be7eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:13.763385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.764008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.766023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1213/2421] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:13 2022] DELETE /identity/v3/projects/0285d8367ffa49fe8aa1fed079be7eb0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:13.770129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:13.771589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:13.784575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba32788-5d1e-489c-82bf-dbe9adba47b8 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:13.840284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:13.840939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:13.841220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:13.841494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] PATH_INFO: `/v3/users/eb2e0f8627fc4603a10b5ec0b9a1642b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:13.842603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2e0f8627fc4603a10b5ec0b9a1642b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:13.844546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:13.846764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:13.894938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2e0f8627fc4603a10b5ec0b9a1642b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.902219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2e0f8627fc4603a10b5ec0b9a1642b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.914108 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2e0f8627fc4603a10b5ec0b9a1642b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.914741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2e0f8627fc4603a10b5ec0b9a1642b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.921554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-3d99c920-7b7e-45d6-8409-13fecc67e7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2e0f8627fc4603a10b5ec0b9a1642b'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:13.927695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1209/2422] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:13 2022] DELETE /identity/v3/users/eb2e0f8627fc4603a10b5ec0b9a1642b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:14.193243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.194739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:14.209031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e5dd4-00e3-4cec-9323-2384435b849a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.267588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.268244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.268555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.268827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] PATH_INFO: `/v3/projects/7a1996c50eb5483f9cb85a761bed2aec` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.269950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1996c50eb5483f9cb85a761bed2aec)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.271849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.318537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1996c50eb5483f9cb85a761bed2aec', 'request_id': 'req-bd17479c-d1d5-4d95-acf8-5057467baa49'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:14.326521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1996c50eb5483f9cb85a761bed2aec', 'request_id': 'req-bd17479c-d1d5-4d95-acf8-5057467baa49'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:14.332724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1996c50eb5483f9cb85a761bed2aec', 'request_id': 'req-bd17479c-d1d5-4d95-acf8-5057467baa49'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:14.344149 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] The token cache is being invalidate because project 7a1996c50eb5483f9cb85a761bed2aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:14.344469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:14.345123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bd17479c-d1d5-4d95-acf8-5057467baa49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:14.346614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1214/2423] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:14 2022] DELETE /identity/v3/projects/7a1996c50eb5483f9cb85a761bed2aec => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:14.352828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.353185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.353529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.373464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:14.375240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:14.438150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1210/2424] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:14.447163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.447470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.447741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.468490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:14.469600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:14.477839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1215/2425] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:14.482046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.492278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcfe0-f720-4baf-afe0-0572eaea98ce None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.496830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.497425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.497703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.498004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.498638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:35:14.498936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.501156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.513173 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1211/2426] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:35:14.517659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.527273 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9d344-4d30-4a2d-a86f-1ad5102e7528 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.532877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.533554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.533944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.534221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.534849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.536753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.567687 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1216/2427] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:14.572076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.582544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0f2e-f576-434a-8e46-3724a2f45437 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.643123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.643740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.644010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.644287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.644977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.646908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.665589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:14.672065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1212/2428] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:35:14.675915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.686561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0023c-9228-4c02-b956-61ed503c5528 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.691307 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.691930 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.692206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.692515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.693155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.694766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.705690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1217/2429] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:14.709833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.721333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebe0da-dc67-4af7-959c-d02396937f65 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.726393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.727012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.727277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.727811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.728493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.730079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d942aae2-57f4-47e8-b81e-d0f62933ab3b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.742152 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1213/2430] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:14.746555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.757022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ba6cf-b0a8-4cab-8124-d61709435aa4 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.761677 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.762325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.762600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.762867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] PATH_INFO: `/v3/projects/063d324768314fc484ebe6a398398361/users/aeeeca581eda41a187ff5587bac50c16/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.772038 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:14.780865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063d324768314fc484ebe6a398398361, user_id=aeeeca581eda41a187ff5587bac50c16, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.783891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:14.799553 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1218/2431] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:14 2022] PUT /identity/v3/projects/063d324768314fc484ebe6a398398361/users/aeeeca581eda41a187ff5587bac50c16/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:14.810864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.811347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.811458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.838829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] MFA Rules not processed for user `aeeeca581eda41a187ff5587bac50c16`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:14.840041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:14.909380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1214/2432] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:14.913652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:14.975120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef217c-d2ca-4965-989a-f10c934d1797 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:14.981009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:14.981622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:14.981896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:14.982162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:14.982799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:14.985198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.015612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1219/2433] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:14 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:15.016782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.028131 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560566a0-bd72-4b24-87d2-902ae68365f7 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.087137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.087742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.088018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.088283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.088935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.090833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.122510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1215/2434] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:15 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:15.124165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.137985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085767dc-1c9c-427e-968a-b9cc0590afa5 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.143439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.144078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.144392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.144668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.145315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.146914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.163091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1220/2435] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:15.164250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.174013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499eab2-97f6-4e9a-bea9-e1955b01115f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.178748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.179394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.179669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.179942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] PATH_INFO: `/v3/projects/e06e63ce2b384d39a5e3ba64998b385d/users/e2ab9bfd08ca4bea8600b0ca7015c0e3/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.196059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06e63ce2b384d39a5e3ba64998b385d, user_id=e2ab9bfd08ca4bea8600b0ca7015c0e3, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.198701 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.212638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1216/2436] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:15 2022] PUT /identity/v3/projects/e06e63ce2b384d39a5e3ba64998b385d/users/e2ab9bfd08ca4bea8600b0ca7015c0e3/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:15.216572 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.227167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4d647-d0fa-45c2-a3aa-8eb2d936061f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.284838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.285458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.285736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.286003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.286650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.288305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53257c8-3c3d-43ba-8a06-1f4acdb0d4f6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.298402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1221/2437] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:15.302658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.313665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded1893-9ec5-4933-9ab1-e17a2f067f13 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.318232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.318877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.319352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.319629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] PATH_INFO: `/v3/projects/e06e63ce2b384d39a5e3ba64998b385d/users/e2ab9bfd08ca4bea8600b0ca7015c0e3/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.321558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e06e63ce2b384d39a5e3ba64998b385d, user_id=e2ab9bfd08ca4bea8600b0ca7015c0e3, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.324174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.335097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1217/2438] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:15 2022] PUT /identity/v3/projects/e06e63ce2b384d39a5e3ba64998b385d/users/e2ab9bfd08ca4bea8600b0ca7015c0e3/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:15.347205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.347582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.347922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.376830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] MFA Rules not processed for user `e2ab9bfd08ca4bea8600b0ca7015c0e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:15.378195 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:15.423283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:15.448232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1222/2439] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:15.459879 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:15.461611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:15.494165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:15.521759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01615210-ee2b-4d51-bcd7-c284ed816952 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:15.525461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:15.526055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:15.526331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:15.526600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:15.536013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:15.537483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b980cd60-3b70-4697-83d1-2ca3762ca3d6 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:15.547495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1218/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:16.223271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:16.291077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3df878-d24d-4644-a3d2-1ab941088404 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:16.295751 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:16.297017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:16.297017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:16.297017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] PATH_INFO: `/v3/users/aeeeca581eda41a187ff5587bac50c16` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:16.298207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeeeca581eda41a187ff5587bac50c16)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:16.300132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:16.355360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeeeca581eda41a187ff5587bac50c16'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.366630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeeeca581eda41a187ff5587bac50c16'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.367229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeeeca581eda41a187ff5587bac50c16'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.376278 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeeeca581eda41a187ff5587bac50c16'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.389231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeeeca581eda41a187ff5587bac50c16'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.397550 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1223/2441] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:16 2022] DELETE /identity/v3/users/aeeeca581eda41a187ff5587bac50c16 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:16.402835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:16.402835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:16.402835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:16.423018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:16.424424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:16.490611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1219/2442] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:16.503401 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:16.505326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:16.570454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6f9b4-80f9-4582-b7ca-8d955fb65f74 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:16.574277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:16.574916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:16.575191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:16.575460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:16.586206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:16.587655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:16.600714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1224/2443] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:16.885439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:16.887018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:16.898789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca0fd4-7b97-482a-a908-95a48a05c076 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:16.903478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:16.904252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:16.904559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:16.904850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] PATH_INFO: `/v3/projects/063d324768314fc484ebe6a398398361` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:16.906960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] RBAC: Authorizing `identity:delete_project(project_id=063d324768314fc484ebe6a398398361)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:16.909350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:16.962318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063d324768314fc484ebe6a398398361', 'request_id': 'req-4ee34b28-73d9-4173-83ac-f1c4152d784a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.970749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063d324768314fc484ebe6a398398361', 'request_id': 'req-4ee34b28-73d9-4173-83ac-f1c4152d784a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.977214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063d324768314fc484ebe6a398398361', 'request_id': 'req-4ee34b28-73d9-4173-83ac-f1c4152d784a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.989140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] The token cache is being invalidate because project 063d324768314fc484ebe6a398398361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:16.989435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.990066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:16.991518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1220/2444] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:16 2022] DELETE /identity/v3/projects/063d324768314fc484ebe6a398398361 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:16.995514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.001509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:17.005555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75459bec-997e-4d47-8be7-5c72fb356e6e service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.062311 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.062935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.063206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.063473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] PATH_INFO: `/v3/users/e2ab9bfd08ca4bea8600b0ca7015c0e3` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.064598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2ab9bfd08ca4bea8600b0ca7015c0e3)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.066501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.111900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2ab9bfd08ca4bea8600b0ca7015c0e3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.118975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2ab9bfd08ca4bea8600b0ca7015c0e3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.119590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2ab9bfd08ca4bea8600b0ca7015c0e3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.126461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2ab9bfd08ca4bea8600b0ca7015c0e3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.138245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-cafebe50-1dff-4644-834f-87063e97290e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2ab9bfd08ca4bea8600b0ca7015c0e3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.143918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1225/2445] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:16 2022] DELETE /identity/v3/users/e2ab9bfd08ca4bea8600b0ca7015c0e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:17.404714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.406762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:17.425912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee34b28-73d9-4173-83ac-f1c4152d784a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.476097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:17.483893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.484862 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.485196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.485467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] PATH_INFO: `/v3/projects/e06e63ce2b384d39a5e3ba64998b385d` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.486921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e06e63ce2b384d39a5e3ba64998b385d)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.489059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.533728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e06e63ce2b384d39a5e3ba64998b385d', 'request_id': 'req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.540572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e06e63ce2b384d39a5e3ba64998b385d', 'request_id': 'req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.546692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e06e63ce2b384d39a5e3ba64998b385d', 'request_id': 'req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.557260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] The token cache is being invalidate because project e06e63ce2b384d39a5e3ba64998b385d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:17.557545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.558170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dcbd6d61-5eef-41a4-8d58-f0b827c649cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:17.559608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1221/2446] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:17 2022] DELETE /identity/v3/projects/e06e63ce2b384d39a5e3ba64998b385d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:35:17.564462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.564945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.565040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.585129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:17.586344 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:17.644114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1226/2447] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:17.651686 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.651904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.652175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.672867 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:17.674052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:17.681768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1222/2448] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:17.685752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.695264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf64b8-18eb-496a-b56a-afdedbe392ed None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.699791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.700407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.700685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.700955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.701580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:35:17.701875 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.707825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.720588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1227/2449] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:35:17.724855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.734755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb9090-0420-42bc-9f4e-fdd9ee21026b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.739297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.739884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.740158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.740464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.741109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.742971 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.774432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1223/2450] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:17 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:35:17.778694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.789056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f8a40-6a6a-40d9-a28c-c0602c417186 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.797423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:17.850385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.851053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.851342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.851608 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.852228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.854408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.876786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1228/2451] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:17 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:17.881019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.891610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6929b1a4-e45a-44d0-b8d4-f8f212856eb5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.896156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.896785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.897062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.897336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.897961 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.899533 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.910041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1224/2452] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:17.914027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.923787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845267f-c706-4147-9243-550d7b7a2a56 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.928353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.928934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.929207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.929500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.930148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.931814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34e4a5-05ea-4654-92cb-6bbb0b000e0a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.942141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1229/2453] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:17.946187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:17.956681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f133b08-f173-406f-9813-ed623453bd6b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:17.961376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:17.962005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:17.962282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:17.962552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] PATH_INFO: `/v3/projects/3cf39a29e2224384a85c8e025d20863b/users/16ff61dafd24462faba19e09f1ca2769/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:17.977093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cf39a29e2224384a85c8e025d20863b, user_id=16ff61dafd24462faba19e09f1ca2769, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:17.979964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:17.996462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1225/2454] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:17 2022] PUT /identity/v3/projects/3cf39a29e2224384a85c8e025d20863b/users/16ff61dafd24462faba19e09f1ca2769/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:18.006817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.007102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.007376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.035212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] MFA Rules not processed for user `16ff61dafd24462faba19e09f1ca2769`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:18.036455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:18.099402 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1230/2455] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:18.104154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.165819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346be62-d4c6-4f7e-8f3c-52f0d08dee71 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.170264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.170861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.171140 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.171413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.172045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.173973 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.199819 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1226/2456] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:18 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:18.204437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.214756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbd910-3b83-4e60-b59f-f179d178b6ab None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.269423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.270016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.270295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.270560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.271167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.273086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.293803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1231/2457] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:18 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:18.297759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.307753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c9b9a-7661-4b33-9090-62d781238971 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.312833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.313415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.313694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.313964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.314589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.316175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.317379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:18.329794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1227/2458] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:18.333689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.343546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8777f1c-a75e-44d7-a960-a2861a603303 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.348055 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.348713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.348985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.349253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] PATH_INFO: `/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d/users/f03657f3632c424186e9cecf332177b2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.352161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:18.364753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d235e6cf3c2f4d6484cca1e083bf2c1d, user_id=f03657f3632c424186e9cecf332177b2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.367398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.379807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1232/2459] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:18 2022] PUT /identity/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d/users/f03657f3632c424186e9cecf332177b2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:18.383768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.394342 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b105ee-ddb7-49e8-9f51-7d8cb97e811c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.449133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.449755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.449931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.450206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.450835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.452459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1259c45-ce55-4ad9-9888-1d11d5654828 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.462645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1228/2460] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:18.466896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.476969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30adc9-9e48-4b5c-8596-ae716a581d72 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.481705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.482337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.482607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.482870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] PATH_INFO: `/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d/users/f03657f3632c424186e9cecf332177b2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.484776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d235e6cf3c2f4d6484cca1e083bf2c1d, user_id=f03657f3632c424186e9cecf332177b2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.487360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.497178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1233/2461] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:18 2022] PUT /identity/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d/users/f03657f3632c424186e9cecf332177b2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:18.508780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.509062 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.509335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.537070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] MFA Rules not processed for user `f03657f3632c424186e9cecf332177b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:18.538260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:18.603918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1229/2462] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:18.614610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:18.616415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:18.674489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0ae41-8705-47c6-b25b-bcdc2593df30 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:18.678145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:18.678741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:18.679017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:18.679286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:18.735501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:18.736939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:18.745573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1234/2463] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:22.949377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:22.950755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:22.963005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9dc04-a813-47ea-9c1a-711c8b08afb4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:22.966518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:22.967107 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:22.967380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:22.967649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:22.976424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:22.978105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208dddc0-49af-4bdd-b190-20f0ad7b2b52 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:22.987277 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1230/2464] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:22 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:27.871425 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:27.934261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccbe8e6-6d93-434d-8fa9-9b6ecdb340c4 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:27.939137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:27.939762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:27.940035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:27.940330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] PATH_INFO: `/v3/users/16ff61dafd24462faba19e09f1ca2769` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:27.941433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16ff61dafd24462faba19e09f1ca2769)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:27.943370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:27.976043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:27.996984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16ff61dafd24462faba19e09f1ca2769'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.004940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16ff61dafd24462faba19e09f1ca2769'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.005656 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16ff61dafd24462faba19e09f1ca2769'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.012962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16ff61dafd24462faba19e09f1ca2769'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.025466 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16ff61dafd24462faba19e09f1ca2769'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.034453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1235/2465] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:27 2022] DELETE /identity/v3/users/16ff61dafd24462faba19e09f1ca2769 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:28.036127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:28.036627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:28.036735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:28.056414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:28.057623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:28.116684 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1231/2466] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:28.126970 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:28.128378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:28.185989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00d2bc0-e7db-4e8d-90a2-2c0f0fe4d620 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:28.190282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:28.190880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:28.191157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:28.191420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:28.200464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:28.201841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:28.211870 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1236/2467] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:28.481396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:28.482848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:28.491724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:28.496206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ed3176-802b-4135-8fdc-dcac79c609a1 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:28.501795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:28.502541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:28.502903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:28.503240 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] PATH_INFO: `/v3/projects/3cf39a29e2224384a85c8e025d20863b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:28.504396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cf39a29e2224384a85c8e025d20863b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:28.506304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:28.551616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cf39a29e2224384a85c8e025d20863b', 'request_id': 'req-da524a3b-b402-41d6-bc54-5f5c21715f47'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.559286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cf39a29e2224384a85c8e025d20863b', 'request_id': 'req-da524a3b-b402-41d6-bc54-5f5c21715f47'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.565322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cf39a29e2224384a85c8e025d20863b', 'request_id': 'req-da524a3b-b402-41d6-bc54-5f5c21715f47'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.575914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] The token cache is being invalidate because project 3cf39a29e2224384a85c8e025d20863b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:28.576201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.576856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.578302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1232/2468] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:28 2022] DELETE /identity/v3/projects/3cf39a29e2224384a85c8e025d20863b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:28.582464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:28.591829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5616b86-309d-4464-a1ef-bf04d338e219 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:28.647025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:28.647650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:28.647925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:28.648201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] PATH_INFO: `/v3/users/f03657f3632c424186e9cecf332177b2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:28.649329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03657f3632c424186e9cecf332177b2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:28.651268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:28.695362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03657f3632c424186e9cecf332177b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.697717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:28.703019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03657f3632c424186e9cecf332177b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.703933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03657f3632c424186e9cecf332177b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.711382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03657f3632c424186e9cecf332177b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.722733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-82c827bd-ac24-48cb-ad75-8f4786086be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03657f3632c424186e9cecf332177b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:28.728692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1237/2469] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:35:28 2022] DELETE /identity/v3/users/f03657f3632c424186e9cecf332177b2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:28.999530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.001144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:29.015607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da524a3b-b402-41d6-bc54-5f5c21715f47 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.072896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.073559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.073780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.074049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] PATH_INFO: `/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.075130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d235e6cf3c2f4d6484cca1e083bf2c1d)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.077069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.122493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd235e6cf3c2f4d6484cca1e083bf2c1d', 'request_id': 'req-c64150d8-19c1-4e47-ada3-267d9ac19f4c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:29.129415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd235e6cf3c2f4d6484cca1e083bf2c1d', 'request_id': 'req-c64150d8-19c1-4e47-ada3-267d9ac19f4c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:29.134902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd235e6cf3c2f4d6484cca1e083bf2c1d', 'request_id': 'req-c64150d8-19c1-4e47-ada3-267d9ac19f4c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:29.146197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] The token cache is being invalidate because project d235e6cf3c2f4d6484cca1e083bf2c1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:35:29.146480 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:29.147099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c64150d8-19c1-4e47-ada3-267d9ac19f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:35:29.148598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1233/2470] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:35:28 2022] DELETE /identity/v3/projects/d235e6cf3c2f4d6484cca1e083bf2c1d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:29.154815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.155275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.155368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.175017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:29.176191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:29.233985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1238/2471] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:29.242483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.242707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.243007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.262242 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:29.263433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:29.271675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1234/2472] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:29.275836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.286007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88baf5b4-573c-4ff6-bc90-e423cfd33eeb None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.290825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.291140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.291414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.291686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.292403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:35:29.292732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.294317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.308764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1239/2473] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:35:29.313258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.322458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fd587-3006-4310-bbf9-fe11d8e83160 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.327147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.327742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.328023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.328316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.329522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.331447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.340742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:29.360243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1235/2474] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:29 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:29.364636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.374557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57113a-3ba6-464a-b0dd-511ef095389c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.428892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.429483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.429755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.430022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.430746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.432676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.452851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1240/2475] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:29 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:29.457517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.468255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374d6d5-e19e-413c-8473-904e8f5b9599 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.473636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.474236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.474518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.474790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.475431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.477091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.491683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1236/2476] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:35:29.495715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.506296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99021ce-6c10-4a6a-b7a7-ec07a1ca28c6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.510809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.511392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.511663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.511929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.512600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.514158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636cdd74-eaee-4700-becd-dc4fc904016a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.524920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1241/2477] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:29.528798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.538498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf626794-75b7-4082-8e09-4827f92970fa admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.543554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.544160 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.544535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.544812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] PATH_INFO: `/v3/projects/5d44fc4fa85a4dfb82c31a602037a249/users/f42af00458ac4046888a73397218f9f5/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.558750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d44fc4fa85a4dfb82c31a602037a249, user_id=f42af00458ac4046888a73397218f9f5, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.561384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.574419 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1237/2478] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:29 2022] PUT /identity/v3/projects/5d44fc4fa85a4dfb82c31a602037a249/users/f42af00458ac4046888a73397218f9f5/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:29.579852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49accdc7-0d25-4ec5-99a6-7e65b3f6783d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.580127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49accdc7-0d25-4ec5-99a6-7e65b3f6783d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.580431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49accdc7-0d25-4ec5-99a6-7e65b3f6783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.598682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-49accdc7-0d25-4ec5-99a6-7e65b3f6783d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:29.600399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-49accdc7-0d25-4ec5-99a6-7e65b3f6783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:29.633889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:29.666464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1242/2479] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:29.676732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:29.678415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:29.735593 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e307f7-0c5b-4cc0-bfa4-8b71ff931d6a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:29.739185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:29.739770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:29.740040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:29.740323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:29.749066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:29.750400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:29.760029 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1238/2480] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:33.639086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:33.639640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:33.639640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:33.669615 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] MFA Rules not processed for user `f42af00458ac4046888a73397218f9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:33.670923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:33.735001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1243/2481] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:35:33.739441 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:33.749700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acccce8-8532-4566-92d1-1f50aa00d02d service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:33.754406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:33.755006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:33.755281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:33.755550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:33.756188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:33.758110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:33.800356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1239/2482] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:35:33 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:33.800753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:33.815353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861a7f4-eb5a-4995-b4fa-ebf79a5a95f6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:33.886021 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:33.886663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:33.886943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:33.887205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:33.887831 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:33.889857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:33.914357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1244/2483] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:35:33 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:35:33.918327 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:33.929647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1ef4d1-d59a-4a5a-953c-fc9ca8c04df3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:33.934708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:33.935325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:33.935595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:33.935861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:33.936536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:33.938161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:33.955362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1240/2484] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:33.956494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:33.966601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d174b46-ef5d-4549-ae12-05dcd5e0ab10 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:33.972691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:33.973447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:33.973794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:33.974070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] PATH_INFO: `/v3/projects/51980456e4f84b9183583477f03f3731/users/84eda3abf950456684a21245bbf17c55/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:33.989872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=51980456e4f84b9183583477f03f3731, user_id=84eda3abf950456684a21245bbf17c55, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:33.993266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:34.008389 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1245/2485] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:33 2022] PUT /identity/v3/projects/51980456e4f84b9183583477f03f3731/users/84eda3abf950456684a21245bbf17c55/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:34.012554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:34.023374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6d089-1e80-49cc-a4bc-0b189da97f02 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:34.082285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:34.082892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:34.083169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:34.083438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:34.084069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:34.085729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a2c31-c175-4809-bb65-d35e7043c430 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:34.096360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1241/2486] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:35:34.100759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:34.111644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11ba27-c97e-4c0d-b45b-6dcc858325fd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:34.116397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:34.117056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:34.117331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:34.117692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] PATH_INFO: `/v3/projects/51980456e4f84b9183583477f03f3731/users/84eda3abf950456684a21245bbf17c55/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:34.119497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51980456e4f84b9183583477f03f3731, user_id=84eda3abf950456684a21245bbf17c55, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:34.122160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:34.133376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1246/2487] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:35:34 2022] PUT /identity/v3/projects/51980456e4f84b9183583477f03f3731/users/84eda3abf950456684a21245bbf17c55/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:35:38.390384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:38.390749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:38.390749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:38.433487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] MFA Rules not processed for user `84eda3abf950456684a21245bbf17c55`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:38.434708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:38.444983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:38.513804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1242/2488] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:35:38.531062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:38.533904 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:38.554041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:38.612152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161bb34-24c2-4c63-a9e6-9814a07e0408 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:38.617618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:38.618218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:38.618492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:38.618758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:38.676507 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:38.678148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052eb0f-ac79-4703-8337-cad7ff1b5445 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:38.687730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1247/2489] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:40.261545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:35:40.277871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cb1f7-a48b-4cef-b212-ca0f22fa33fd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:35:40.284617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:35:40.285294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:40.285566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:40.285869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:40.287702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:40.394764 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:35:40.397434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e437b-81e0-4879-beb4-146dfa628f2c service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:35:40.410232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1243/2490] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:35:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:35:48.683014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=49 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:50.404548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=53 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:35:55.220201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:35:55.220738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:35:55.220738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:35:55.239545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] MFA Rules not processed for user `f42af00458ac4046888a73397218f9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:35:55.240768 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:35:55.247574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1248/2491] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:35:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:05.234168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:05.766713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:05.767218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:05.767319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:05.786612 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:05.787813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:05.842357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1244/2492] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:05.895481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:05.897059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:05.989334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214efa28-bb18-4b27-a18d-8516195f8460 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:05.999043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:05.999934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:06.000219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:06.001103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:06.009935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:06.011558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:06.021247 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1249/2493] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:36:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:36:08.434940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:08.505053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8073c73-32b7-441e-9e41-a2e25177d806 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:08.509539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:08.510289 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:08.510813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:08.511147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:08.512936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:08.586584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:08.588323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:08.600793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1245/2494] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:14.092437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:14.094087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:14.111580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0abec32-fa85-498b-83a8-7f440c00003d service swift] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:14.116647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:14.117487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:14.117925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:14.118343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] PATH_INFO: `/v3/users/49c772b0c2bb49a4ab4048d980bd20a8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:14.119727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c772b0c2bb49a4ab4048d980bd20a8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:14.121986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:14.201819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c772b0c2bb49a4ab4048d980bd20a8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:14.213725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c772b0c2bb49a4ab4048d980bd20a8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:14.214570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c772b0c2bb49a4ab4048d980bd20a8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:14.226077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c772b0c2bb49a4ab4048d980bd20a8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:14.247575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-367aee6a-4814-4111-a426-7351eef9c1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c772b0c2bb49a4ab4048d980bd20a8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:14.259295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1250/2495] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:14 2022] DELETE /identity/v3/users/49c772b0c2bb49a4ab4048d980bd20a8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:14.715962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:14.735118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea12732e-c609-4598-818e-f3f1f2e5edd9 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:14.866840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:14.872437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:14.872643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:14.872912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:14.873182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] PATH_INFO: `/v3/projects/89fe74d1d646452e83cb93420766c0a2` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:14.874370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=89fe74d1d646452e83cb93420766c0a2)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:14.877461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:14.988006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89fe74d1d646452e83cb93420766c0a2', 'request_id': 'req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:15.006986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89fe74d1d646452e83cb93420766c0a2', 'request_id': 'req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:15.019205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89fe74d1d646452e83cb93420766c0a2', 'request_id': 'req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:15.057974 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] The token cache is being invalidate because project 89fe74d1d646452e83cb93420766c0a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:15.058272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:15.058945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-1cf2e54a-8f99-4c8b-9fbc-005e12568e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:15.060715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1246/2496] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:14 2022] DELETE /identity/v3/projects/89fe74d1d646452e83cb93420766c0a2 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:15.066099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.066470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.066818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.095170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:15.097181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:15.296670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1251/2497] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:15.311043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.311043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.311043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.366916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:15.366916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:15.385794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1247/2498] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:15.390452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.404517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:15.421112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b115188-b2bd-4381-9961-edfccbd91ff7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.440282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.441081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.443880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.444241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.445037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:15.445416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.447131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.478601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1252/2499] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:15.483580 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.518951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06abdb-8c15-4170-bb84-7714d5ab53f9 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.541484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.544647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.546063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.548123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.550116 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.558919 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.634789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1248/2500] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:15 2022] POST /identity/v3/projects => generated 363 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:15.639192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.673827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181e9a6-410c-453a-a2cf-4e2c0dac3654 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.786736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.787495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.787874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.788249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.789022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.791081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.827795 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1253/2501] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:15 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:15.828549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.843752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98f6f-f3bf-44cb-bcac-42187f55ba93 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.851769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.852430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.852966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.853244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.853884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.855652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.879576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1249/2502] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:15.883421 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.894497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59a284-8246-477d-af38-21bcb05f423a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.899505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.900133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.900468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.900736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.901395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.903017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c607e4c-13c6-4f46-83a7-8c0e049b002e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.915689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1254/2503] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:15.919785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:15.932907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c1bfd4-ee4f-491e-8f48-a0d1da81abc3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:15.940334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:15.941228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.941228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.941228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] PATH_INFO: `/v3/projects/85a8d81c09a045009337248488ba4d36/users/3cde45fd134244f68a062ece3179c634/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:15.957297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a8d81c09a045009337248488ba4d36, user_id=3cde45fd134244f68a062ece3179c634, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:15.960628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:15.978183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1250/2504] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:15 2022] PUT /identity/v3/projects/85a8d81c09a045009337248488ba4d36/users/3cde45fd134244f68a062ece3179c634/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:15.991225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:15.991225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:15.991225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.027564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] MFA Rules not processed for user `3cde45fd134244f68a062ece3179c634`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:16.029122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:16.158629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1255/2505] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:16.163507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.263372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62b44a-37a0-4f47-bae9-ee18c9072f8f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.274665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.275473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.275846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.276243 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.276997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.279183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.315161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1251/2506] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:16 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:16.318991 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.333834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df67b29-d210-4298-8c7e-c2af23a09c2d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.402856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:16.416684 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.417495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.417868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.418209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.418978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.421057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.451878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1256/2507] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:16 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:16.456595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.468570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da90bd09-8cfe-4f4f-86a8-7656097c20bc admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.474541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.475338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.475912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.476285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.477083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.479055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.480768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:16.497993 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1252/2508] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:16.500971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.514635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606d78d-ec61-429e-ba4e-2fad1b0a0ea2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.520536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.521352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.521654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.521960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] PATH_INFO: `/v3/projects/11df04d14d8b4404a9fe99b82d5ea416/users/b042743a01ad40bfa2f32cb6e1c4e2a7/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.539128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] RBAC: Authorizing `identity:create_grant(project_id=11df04d14d8b4404a9fe99b82d5ea416, user_id=b042743a01ad40bfa2f32cb6e1c4e2a7, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.541851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.558439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1257/2509] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:16 2022] PUT /identity/v3/projects/11df04d14d8b4404a9fe99b82d5ea416/users/b042743a01ad40bfa2f32cb6e1c4e2a7/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:16.562827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.577503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c302d2-d305-4c53-92cf-960a68c13558 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.681782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.682489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.682959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.683179 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.683881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.685762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06b4e3-dff8-4971-adf1-026d89541ea5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.698380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1253/2510] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:16.702698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.716770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09fc23-f250-4055-b6e5-10f14e1fcacd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:16.729665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:16.730355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.733606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.734194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] PATH_INFO: `/v3/projects/11df04d14d8b4404a9fe99b82d5ea416/users/b042743a01ad40bfa2f32cb6e1c4e2a7/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.737375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=11df04d14d8b4404a9fe99b82d5ea416, user_id=b042743a01ad40bfa2f32cb6e1c4e2a7, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:16.740607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:16.754029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1258/2511] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:16 2022] PUT /identity/v3/projects/11df04d14d8b4404a9fe99b82d5ea416/users/b042743a01ad40bfa2f32cb6e1c4e2a7/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:16.764265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:16.764588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:16.764869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:16.796260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] MFA Rules not processed for user `b042743a01ad40bfa2f32cb6e1c4e2a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:16.798105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:16.884350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1254/2512] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:16.894495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:16.896861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:17.011124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e5b0e9-f569-462e-acbc-c29359af6c8f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:17.016130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:17.017342 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:17.017958 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:17.018442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:17.034071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:17.035902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:17.051211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1259/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:17.191064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:17.202283 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722d973-1d77-424d-b9e2-cfdc0ebad117 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:17.214041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:17.216232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:17.216718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:17.217169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:17.225373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:17.227180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:17.331178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:17.332812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:17.344538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5d646-d759-4ae5-94f3-4b34f0a28e46 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:17.350156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:17.351611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:17.352109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:17.352934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:17.355963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:17.356414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1255/2514] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:17.470243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:17.473028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd30147-a7f4-4591-8a89-fa10837b5a76 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:17.510490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1260/2515] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:17 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:20.259403 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:20.290747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:20.345343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f305e-86f7-4545-9158-7f881c69d4eb service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:20.349858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:20.350495 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:20.350945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:20.351218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] PATH_INFO: `/v3/users/3cde45fd134244f68a062ece3179c634` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:20.352404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cde45fd134244f68a062ece3179c634)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:20.355137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:20.431511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cde45fd134244f68a062ece3179c634'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:20.440726 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cde45fd134244f68a062ece3179c634'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:20.460060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cde45fd134244f68a062ece3179c634'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:20.460060 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cde45fd134244f68a062ece3179c634'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:20.466588 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cde45fd134244f68a062ece3179c634'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:20.478344 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1256/2516] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:20 2022] DELETE /identity/v3/users/3cde45fd134244f68a062ece3179c634 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:20.479761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:20.480225 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:20.480337 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:20.488175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:20.502487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:20.503699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:20.580065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1261/2517] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:20.589719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:20.591439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:20.671598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b695bb-6c78-4c70-90ca-b0a23c58bcc7 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:20.676417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:20.677133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:20.678787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:20.679090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:20.690325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:20.691887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:20.703545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1257/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:21.016707 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:21.018834 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:21.034523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a120c-2d62-47cd-948c-0e0805869aa9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:21.040975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:21.041639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.041916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.042187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] PATH_INFO: `/v3/projects/85a8d81c09a045009337248488ba4d36` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.043295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a8d81c09a045009337248488ba4d36)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:21.047270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:21.103393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a8d81c09a045009337248488ba4d36', 'request_id': 'req-71a91d75-39de-4e49-9614-c192ac0861a3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.112702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a8d81c09a045009337248488ba4d36', 'request_id': 'req-71a91d75-39de-4e49-9614-c192ac0861a3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.119264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a8d81c09a045009337248488ba4d36', 'request_id': 'req-71a91d75-39de-4e49-9614-c192ac0861a3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.130513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] The token cache is being invalidate because project 85a8d81c09a045009337248488ba4d36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:21.130829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.131515 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.133054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1262/2519] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:21 2022] DELETE /identity/v3/projects/85a8d81c09a045009337248488ba4d36 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:21.137144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:21.147323 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e54658-3638-43da-aacf-db49b136295c service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:21.181036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:21.209435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:21.210071 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.210346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.210618 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] PATH_INFO: `/v3/users/b042743a01ad40bfa2f32cb6e1c4e2a7` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.212123 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b042743a01ad40bfa2f32cb6e1c4e2a7)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:21.214225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:21.266816 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b042743a01ad40bfa2f32cb6e1c4e2a7'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.277110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b042743a01ad40bfa2f32cb6e1c4e2a7'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.296206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b042743a01ad40bfa2f32cb6e1c4e2a7'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.296995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b042743a01ad40bfa2f32cb6e1c4e2a7'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.304628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-4607f2d3-d482-4a03-b3af-428e0f441d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b042743a01ad40bfa2f32cb6e1c4e2a7'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.311122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1258/2520] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:21 2022] DELETE /identity/v3/users/b042743a01ad40bfa2f32cb6e1c4e2a7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:21.592936 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:21.594447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:21.617982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a91d75-39de-4e49-9614-c192ac0861a3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:21.705207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:21.705890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.706319 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.706669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] PATH_INFO: `/v3/projects/11df04d14d8b4404a9fe99b82d5ea416` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.709223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11df04d14d8b4404a9fe99b82d5ea416)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:21.712621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:21.786749 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:21.811108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11df04d14d8b4404a9fe99b82d5ea416', 'request_id': 'req-7ede30ed-ed0c-49f5-93ea-635b25a93f40'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.821828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11df04d14d8b4404a9fe99b82d5ea416', 'request_id': 'req-7ede30ed-ed0c-49f5-93ea-635b25a93f40'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.829085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11df04d14d8b4404a9fe99b82d5ea416', 'request_id': 'req-7ede30ed-ed0c-49f5-93ea-635b25a93f40'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.850306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] The token cache is being invalidate because project 11df04d14d8b4404a9fe99b82d5ea416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:21.850596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.851259 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:21.856527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1263/2521] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:21 2022] DELETE /identity/v3/projects/11df04d14d8b4404a9fe99b82d5ea416 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:21.858220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed57d3-89b7-4a8b-8cf0-9f630a3d3a1c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.858575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed57d3-89b7-4a8b-8cf0-9f630a3d3a1c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.858920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed57d3-89b7-4a8b-8cf0-9f630a3d3a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.886355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f0ed57d3-89b7-4a8b-8cf0-9f630a3d3a1c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:21.888220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f0ed57d3-89b7-4a8b-8cf0-9f630a3d3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:21.905514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:21.907137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:21.974452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1259/2522] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:21.983330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.983616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.983891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.985004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede30ed-ed0c-49f5-93ea-635b25a93f40 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:21.990101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:21.990713 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:21.990989 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:21.991257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:21.993009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:22.005558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:22.006818 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:22.017138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1260/2523] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:22.017413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.019028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.021230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.034776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1264/2524] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:22.036598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c627a-c2ae-4762-8129-77ffd983f3e5 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.041822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.042522 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.042894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.043235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.043999 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:22.044424 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.046083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.059129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1261/2525] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:22.063622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.074066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45da8a0-1880-4eb5-a155-01c5bd0a0dee service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.079098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.079804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.080146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.080665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.081350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.083494 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.129659 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.135992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1265/2526] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:22 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:22.140345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d62cc-fc5a-4351-abdd-1300d826eb90 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.194975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:22.235621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.236932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.237273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.237611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.238304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.240382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.264796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1262/2527] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:22 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:22.268944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.281530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3a7f4-7864-493d-8497-f16398009f71 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.287139 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.287855 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.288511 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.288850 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.289717 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.291306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.304024 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1266/2528] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:22.308381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.320398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0555e5-f7d6-4d6a-bbbd-f701982507fb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.326731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.327474 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.327914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.328251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.329010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.331390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251de0d9-78a3-4d8a-9641-552c02f47360 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.343846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1263/2529] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:22.348791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.360025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e0903-cfcf-4984-be11-0a59487f041b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.365212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.365874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.366175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.366443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] PATH_INFO: `/v3/projects/c375d17b40544015bde4b2c0c10abda2/users/d67c67a6d7de42ae8ceb487ea5ebbaba/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.383093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c375d17b40544015bde4b2c0c10abda2, user_id=d67c67a6d7de42ae8ceb487ea5ebbaba, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.385782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.403378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1267/2530] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:22 2022] PUT /identity/v3/projects/c375d17b40544015bde4b2c0c10abda2/users/d67c67a6d7de42ae8ceb487ea5ebbaba/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:22.414376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.414658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.415085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.446213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] MFA Rules not processed for user `d67c67a6d7de42ae8ceb487ea5ebbaba`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:22.447496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:22.523076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1264/2531] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:22.527692 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.596926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2579cf-c8b3-4302-b163-4d66c0ae53ac admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.601798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.602393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.603223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.603490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.604190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.606211 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.612648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:22.641328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1268/2532] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:22 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:22.646437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.660008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834eb663-d8f6-4a63-9344-3b6113a23ee5 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.726280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.726935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.727408 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.727942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.728803 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.730781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.756349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1265/2533] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:22 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:22.759025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.771516 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58402973-2339-4c83-99ac-4729bfde976e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.779028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.780176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.780595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.780978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.781733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.783415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.796974 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1269/2534] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:22.798138 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.812043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517861e-5872-47d8-93e6-d20734dc1fa3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.828523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.828523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.828523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.828523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] PATH_INFO: `/v3/projects/0b5808eeada346239c7cdc5b12320aac/users/6b96cc536a164df8ad99474d00bac338/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.848845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5808eeada346239c7cdc5b12320aac, user_id=6b96cc536a164df8ad99474d00bac338, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.851654 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.877889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1266/2535] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:22 2022] PUT /identity/v3/projects/0b5808eeada346239c7cdc5b12320aac/users/6b96cc536a164df8ad99474d00bac338/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:22.881818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:22.893317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d375f0d-aa6f-4245-8c32-379f0f6ce7fc admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:22.966548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:22.967249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:22.967489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:22.967823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:22.968912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:22.971052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef990e9e-c2fd-4747-8855-8c9311340801 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:22.987967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1270/2536] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:22.993260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:23.007799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89eac3-137b-4122-b0ae-3ea8d9fe226d None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.008043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ede7d-1580-48de-9315-c8790ad8afc2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:23.008426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89eac3-137b-4122-b0ae-3ea8d9fe226d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.009479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89eac3-137b-4122-b0ae-3ea8d9fe226d None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.012257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1271/2537] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:36:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:36:23.018535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:23.019311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.019545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.019697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.020037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] PATH_INFO: `/v3/projects/0b5808eeada346239c7cdc5b12320aac/users/6b96cc536a164df8ad99474d00bac338/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.020601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.021068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.022253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5808eeada346239c7cdc5b12320aac, user_id=6b96cc536a164df8ad99474d00bac338, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:23.024901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9cd0a0-7cbf-4cfb-b244-0ada9ae252c1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:23.031122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:23.044667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1267/2538] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:22 2022] PUT /identity/v3/projects/0b5808eeada346239c7cdc5b12320aac/users/6b96cc536a164df8ad99474d00bac338/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:23.056200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.056503 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.056902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.067890 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:23.069266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.094804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] MFA Rules not processed for user `6b96cc536a164df8ad99474d00bac338`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:23.096254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.161140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1272/2539] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:36:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:23.173911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:23.175316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.193662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1268/2540] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:23.206186 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:23.206852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.248711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fdd9cd-8920-41f5-a02d-5608b584bb83 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:23.253118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:23.253722 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.254001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.254262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.266002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:23.267475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6328e47-e81b-4aad-8154-9c515e0aa668 service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:23.275751 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803996-62c9-4546-97ea-0e049172bdc4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:23.279468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1273/2541] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:23.280314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:23.281058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.281343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.281611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.294367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:23.296758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce240ad-7239-4410-b7c5-b5612c691799 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:23.311047 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1269/2542] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:23.398366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.398650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.398908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] PATH_INFO: `/` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.400375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1274/2543] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:36:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:36:23.412362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de56-bd93-4273-8cf8-2dca04d31b28 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.412866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de56-bd93-4273-8cf8-2dca04d31b28 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.413236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42de56-bd93-4273-8cf8-2dca04d31b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.443262 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-ba42de56-bd93-4273-8cf8-2dca04d31b28 None None] MFA Rules not processed for user `82a67430b66140e1b8427f2ffb627f42`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:23.444696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ba42de56-bd93-4273-8cf8-2dca04d31b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.518457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1270/2544] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:36:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:23.531609 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:23.533956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.550946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1085a-3c56-4890-a726-ac7e5202bd93 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:23.558002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:23.558753 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.559126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.559276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.564155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:23.568573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:23.570663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b81111-79f8-400a-87c5-5ff9935f8a80 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:23.584747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1275/2545] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:23.690248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1854-8964-4a80-a24c-1a9eb8cbfcad None None] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.690619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1854-8964-4a80-a24c-1a9eb8cbfcad None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.690951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788a1854-8964-4a80-a24c-1a9eb8cbfcad None None] PATH_INFO: `/` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.692537 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1271/2546] 10.222.0.33 () {54 vars in 921 bytes} [Thu Sep 22 12:36:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 12:36:23.698209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.698681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.698762 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.721423 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] MFA Rules not processed for user `6db6f737c3354ca692eb6ee91a496223`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:23.722995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:23.733818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1276/2547] 10.222.0.33 () {58 vars in 1018 bytes} [Thu Sep 22 12:36:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:23.746824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a1854-8964-4a80-a24c-1a9eb8cbfcad None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:23.759504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788a1854-8964-4a80-a24c-1a9eb8cbfcad None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:23.763649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:23.764368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:23.764704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:23.765040 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:23.776023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:23.777558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db7829b-24ea-485e-abd6-b5eb185c1720 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:23.800827 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1272/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:25.114109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:25.184051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45c2357-c445-41a5-ba22-b87565ab0ab1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:25.189118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:25.189815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:25.190160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:25.190639 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] PATH_INFO: `/v3/users/d67c67a6d7de42ae8ceb487ea5ebbaba` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:25.191996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67c67a6d7de42ae8ceb487ea5ebbaba)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:25.193950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:25.243972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67c67a6d7de42ae8ceb487ea5ebbaba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:25.251632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67c67a6d7de42ae8ceb487ea5ebbaba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:25.252399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67c67a6d7de42ae8ceb487ea5ebbaba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:25.259794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67c67a6d7de42ae8ceb487ea5ebbaba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:25.272872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67c67a6d7de42ae8ceb487ea5ebbaba'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:25.279001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1277/2549] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:25 2022] DELETE /identity/v3/users/d67c67a6d7de42ae8ceb487ea5ebbaba => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:25.283912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:25.284422 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:25.284590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:25.304478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:25.305717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:25.323512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:25.374448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1273/2550] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:25.385469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:25.387706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:25.463862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f991f-06c0-4e2c-b903-c60c1ea1d9b9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:25.467215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:25.468358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:25.468358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:25.468566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:25.479868 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:25.481662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:25.494892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1278/2551] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:26.221643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:26.223126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.237252 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569c141-4043-49cb-8fbc-ce43493350d8 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:26.245609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:26.246702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.246702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.246923 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] PATH_INFO: `/v3/users/55a7f7ba0b27479f8d1e68d808637d55` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.248113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] RBAC: Authorizing `identity:delete_user(user_id=55a7f7ba0b27479f8d1e68d808637d55)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:26.250782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:26.325571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55a7f7ba0b27479f8d1e68d808637d55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.334476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:26.336206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.336420 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55a7f7ba0b27479f8d1e68d808637d55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.351948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55a7f7ba0b27479f8d1e68d808637d55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.353018 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55a7f7ba0b27479f8d1e68d808637d55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.354921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f951db9-8ae8-4c63-850e-d6559b8f9d04 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:26.361983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-04631362-6e6b-486b-9616-f236b488361c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55a7f7ba0b27479f8d1e68d808637d55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.371162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1274/2552] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:26 2022] DELETE /identity/v3/users/55a7f7ba0b27479f8d1e68d808637d55 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:26.372221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:26.372984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.373317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.373790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] PATH_INFO: `/v3/projects/c375d17b40544015bde4b2c0c10abda2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.374972 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c375d17b40544015bde4b2c0c10abda2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:26.378535 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:26.382592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:26.452008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c375d17b40544015bde4b2c0c10abda2', 'request_id': 'req-6e4979fd-1ce9-42a0-b7d1-baa674c32645'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.452782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.453241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.453668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.461100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c375d17b40544015bde4b2c0c10abda2', 'request_id': 'req-6e4979fd-1ce9-42a0-b7d1-baa674c32645'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.467738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c375d17b40544015bde4b2c0c10abda2', 'request_id': 'req-6e4979fd-1ce9-42a0-b7d1-baa674c32645'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.475247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:26.476485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.480166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] The token cache is being invalidate because project c375d17b40544015bde4b2c0c10abda2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:26.480489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.481143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.482605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1279/2553] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:26 2022] DELETE /identity/v3/projects/c375d17b40544015bde4b2c0c10abda2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:26.488081 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:26.489681 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.549008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1275/2554] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:36:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:26.559610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4979fd-1ce9-42a0-b7d1-baa674c32645 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:26.560812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:26.562625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.565163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:26.565872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.566157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.566521 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] PATH_INFO: `/v3/users/6b96cc536a164df8ad99474d00bac338` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.567937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b96cc536a164df8ad99474d00bac338)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:26.570096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:26.620172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:26.630297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b96cc536a164df8ad99474d00bac338'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.637574 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b96cc536a164df8ad99474d00bac338'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.638299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b96cc536a164df8ad99474d00bac338'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.645794 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b96cc536a164df8ad99474d00bac338'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.651335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6af70-9482-4419-a914-aa4433bedf8a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:26.656437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:26.657045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.657321 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.657979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b96cc536a164df8ad99474d00bac338'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.658133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.660037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.664230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1280/2555] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:26 2022] DELETE /identity/v3/users/6b96cc536a164df8ad99474d00bac338 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:26.664895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:26.666180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:26.674125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:26.676386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b6487-5c76-4263-b7a2-90df62ee2bde service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:26.680444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:26.692546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1276/2556] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:36:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 12:36:26.749209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af89f5-b7e3-4436-a237-1b8e2f322526 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:26.754493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:26.755157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.755431 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.755701 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] PATH_INFO: `/v3/projects/40ddbbe59b90403fb3ea9212d10727aa` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.756871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ddbbe59b90403fb3ea9212d10727aa)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:26.758787 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:26.832006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ddbbe59b90403fb3ea9212d10727aa', 'request_id': 'req-2cce5393-4a4d-4e48-8151-2379b04d5595'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.848059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ddbbe59b90403fb3ea9212d10727aa', 'request_id': 'req-2cce5393-4a4d-4e48-8151-2379b04d5595'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.868346 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ddbbe59b90403fb3ea9212d10727aa', 'request_id': 'req-2cce5393-4a4d-4e48-8151-2379b04d5595'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.896031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] The token cache is being invalidate because project 40ddbbe59b90403fb3ea9212d10727aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:26.896500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.897412 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-2cce5393-4a4d-4e48-8151-2379b04d5595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:26.900407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1281/2557] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:26 2022] DELETE /identity/v3/projects/40ddbbe59b90403fb3ea9212d10727aa => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:26.905556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139530-a881-424b-8798-6cfe45f0549d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:26.909430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139530-a881-424b-8798-6cfe45f0549d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:26.909912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39139530-a881-424b-8798-6cfe45f0549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:26.952390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-39139530-a881-424b-8798-6cfe45f0549d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:26.953811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-39139530-a881-424b-8798-6cfe45f0549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.151025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1277/2558] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:27.163983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.168107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.168107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.201801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39139530-a881-424b-8798-6cfe45f0549d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.203758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:27.205320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-39139530-a881-424b-8798-6cfe45f0549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.206217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.222147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1282/2559] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:27.223203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.229760 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39139530-a881-424b-8798-6cfe45f0549d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.251998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.252996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.253635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.254092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] PATH_INFO: `/v3/projects/0b5808eeada346239c7cdc5b12320aac` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.255580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0b0a0-aab3-484e-ad7c-6708df150ac7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.256472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5808eeada346239c7cdc5b12320aac)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.265301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.265479 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.266548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.267800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.268537 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.269848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:27.270332 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.272228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.288490 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1283/2560] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:27.292331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.305620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f01dbd-18c4-4649-a865-c2001ed34612 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.311725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.312467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.312848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.313162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.315432 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.318047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.355377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5808eeada346239c7cdc5b12320aac', 'request_id': 'req-798c6a77-f245-4a49-a099-a76ba5abe753'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:27.360772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1284/2561] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/projects => generated 383 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:27.365566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.370371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5808eeada346239c7cdc5b12320aac', 'request_id': 'req-798c6a77-f245-4a49-a099-a76ba5abe753'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:27.382656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5808eeada346239c7cdc5b12320aac', 'request_id': 'req-798c6a77-f245-4a49-a099-a76ba5abe753'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:27.403562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] The token cache is being invalidate because project 0b5808eeada346239c7cdc5b12320aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:27.404454 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:27.405810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-798c6a77-f245-4a49-a099-a76ba5abe753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:27.410199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1278/2562] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:27 2022] DELETE /identity/v3/projects/0b5808eeada346239c7cdc5b12320aac => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:27.417007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.417500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.418094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.446158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:27.448045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.482939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab19146-1c45-4719-b0a9-567d455cfa3a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.495059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.496363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.496999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.497450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.498409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.501652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c20546-e900-4592-bc6f-168cd132ba39 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.512372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:27.515106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:27.534913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1279/2563] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:27.541198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1285/2564] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:27.543194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.543839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.544348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.546044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.551148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.577590 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:27.578830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.580166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800e383-fcae-4978-a7d5-fb647346e619 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.590391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1286/2565] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:27.591769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.593615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.594415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.594792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.595196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.596031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.600943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.601971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88c2e7-06f8-4f00-a21e-63082f7dac6c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.607383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.607917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.608194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.608505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.609155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:27.609459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.611696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.615330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1280/2566] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:27.619360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.630810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7f52f2-0edb-4e42-94f1-5bbf676af553 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.632450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1287/2567] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:27.635324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.636871 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.637613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.637962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.638295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.639128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.641032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.648023 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e739a3-2386-4998-b3a9-299cba03c277 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.652281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1281/2568] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:27.656197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.656726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.657489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.657835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.658178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.658891 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.661044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.681210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e47e56-3ae4-4214-b56a-2e358a767bf9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.690425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.691209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.691692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.692046 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] PATH_INFO: `/v3/projects/862082d6e756429ea5829677b8100dcc/users/9af80c1235db4ebaaf30ba33c1e93d1b/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.696754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1288/2569] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:27.697900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.709891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862082d6e756429ea5829677b8100dcc, user_id=9af80c1235db4ebaaf30ba33c1e93d1b, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.712982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc527f6-7d49-45c3-948f-f03bfcf1ac30 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.729790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1282/2570] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:27 2022] PUT /identity/v3/projects/862082d6e756429ea5829677b8100dcc/users/9af80c1235db4ebaaf30ba33c1e93d1b/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:27.741988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.742353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.742695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.773736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] MFA Rules not processed for user `9af80c1235db4ebaaf30ba33c1e93d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:27.775169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.827645 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7319058-b9f2-4d88-aae4-d64516e38e8f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.851017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1283/2571] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:27.864779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.866401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:27.916453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.917066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.917345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.917614 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.918236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.920326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.951200 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1289/2572] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:27 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:27.956134 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:27.968100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dcdf73-ab00-4346-8911-ad0cfcaa3652 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.974602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:27.975317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:27.975669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:27.976011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:27.976901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:27.979017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:27.992041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1290/2573] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:27.993768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f314ab5-a65e-4f53-8530-a28b8f4a37df None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:27.995733 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.000623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.001349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.001632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.001977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.009219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5b8db-982c-4f42-8da6-7f82d7c9acfd admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.011893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.013570 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.014719 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.015365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.015629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.015898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.016562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.018559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b0bc-e391-4db2-a829-2f00f792af7e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.023202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1284/2574] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:28.029859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1291/2575] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:28.033838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.053526 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1ac858-6bd2-4543-8ea3-d93a7bc76acc service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.058894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.059782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.060439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.060866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] PATH_INFO: `/v3/projects/1407bcfd8f50495eb4fb5f0ef0966bfd/users/6ec7aa5b21ea469b90c4405927c66fa6/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.079932 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1407bcfd8f50495eb4fb5f0ef0966bfd, user_id=6ec7aa5b21ea469b90c4405927c66fa6, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.083189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.092481 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:28.099391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1285/2576] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:28 2022] PUT /identity/v3/projects/1407bcfd8f50495eb4fb5f0ef0966bfd/users/6ec7aa5b21ea469b90c4405927c66fa6/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:28.110246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.111446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.111446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.148588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] MFA Rules not processed for user `6ec7aa5b21ea469b90c4405927c66fa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:28.150147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:28.162112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:28.312422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1292/2577] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:28.315356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.440347 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1cb86-a19b-4ca7-9054-9db0e08bacb9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.455063 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.456042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.456397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.456741 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.457443 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.459661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.522092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1286/2578] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:28 2022] POST /identity/v3/projects => generated 387 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:28.523778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.538971 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57b1a3-62fc-4ce3-a0dc-9d41ce27dbde None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.659996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.660747 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.661030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.661371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.662093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.670859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.699651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1293/2579] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:28 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:28.705894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.721718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747692df-165d-429a-b995-baed4779fd55 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.727253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.728338 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.728804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.729478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.730585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.734302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.751640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1287/2580] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:28.752822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.772699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8ec57-c93b-46fb-aac7-6e89d5370276 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.777500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.778188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.778464 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.778738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] PATH_INFO: `/v3/projects/bc42da86bd864d31aec70603c22a5775/users/9e3f4cfbb43f49ccb50327a0f1c766b9/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.803546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc42da86bd864d31aec70603c22a5775, user_id=9e3f4cfbb43f49ccb50327a0f1c766b9, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.806613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.824331 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1294/2581] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:28 2022] PUT /identity/v3/projects/bc42da86bd864d31aec70603c22a5775/users/9e3f4cfbb43f49ccb50327a0f1c766b9/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:28.828496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.843603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b264d-eb3e-4700-ac16-a02fe33bd71e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.930106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.930625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.930849 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.931122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.931759 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.933406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c1f9f-3f5f-443f-b891-d4b129fa5fd9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.944739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1288/2582] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:28.949086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:28.961461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382637d-68cb-4144-ae19-14cfda17239c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:28.967354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:28.968100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.968489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.968820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] PATH_INFO: `/v3/projects/bc42da86bd864d31aec70603c22a5775/users/9e3f4cfbb43f49ccb50327a0f1c766b9/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:28.971525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc42da86bd864d31aec70603c22a5775, user_id=9e3f4cfbb43f49ccb50327a0f1c766b9, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:28.974218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:28.985261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1295/2583] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:28 2022] PUT /identity/v3/projects/bc42da86bd864d31aec70603c22a5775/users/9e3f4cfbb43f49ccb50327a0f1c766b9/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:28.996908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:28.997269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:28.997606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:29.031719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] MFA Rules not processed for user `9e3f4cfbb43f49ccb50327a0f1c766b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:29.032985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:29.107898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:29.120132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1289/2584] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:29.131882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:29.133633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:29.201844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d003-2889-47ab-8c9e-e5d863e96cca admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:29.205848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:29.206463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:29.206744 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:29.207016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:29.217076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:29.218528 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472d4539-6174-40e9-961b-f20d03aa560e service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:29.229012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1296/2585] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:29.301012 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:29.375969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f3ee6-260a-49d6-a486-21924efb3add None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:29.382274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:29.383280 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:29.383562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:29.383836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] PATH_INFO: `/v3/users/6ec7aa5b21ea469b90c4405927c66fa6` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:29.385065 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec7aa5b21ea469b90c4405927c66fa6)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:29.387072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:29.441100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec7aa5b21ea469b90c4405927c66fa6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:29.448247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec7aa5b21ea469b90c4405927c66fa6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:29.467423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec7aa5b21ea469b90c4405927c66fa6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:29.468282 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec7aa5b21ea469b90c4405927c66fa6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:29.477171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec7aa5b21ea469b90c4405927c66fa6'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:29.487609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1290/2586] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:29 2022] DELETE /identity/v3/users/6ec7aa5b21ea469b90c4405927c66fa6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:29.489358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:29.489756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:29.490053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:29.504883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:29.511889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:29.513534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:29.584979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1297/2587] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:29.594100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:29.604767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:29.689518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b4e12-9206-4aa0-ad52-5c8b5cad7f6c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:29.698959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:29.699656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:29.700005 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:29.700401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:29.712532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:29.714291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:29.725789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1291/2588] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:30.148596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:30.150251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:30.171967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14384b6-7aaf-4f6e-9257-0a84046c4586 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:30.181047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:30.181698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:30.181975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:30.182239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] PATH_INFO: `/v3/projects/1407bcfd8f50495eb4fb5f0ef0966bfd` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:30.183487 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1407bcfd8f50495eb4fb5f0ef0966bfd)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:30.185313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:30.260027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1407bcfd8f50495eb4fb5f0ef0966bfd', 'request_id': 'req-49450c2b-d7f8-4b46-b463-69a787752a0e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.268570 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1407bcfd8f50495eb4fb5f0ef0966bfd', 'request_id': 'req-49450c2b-d7f8-4b46-b463-69a787752a0e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.286982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1407bcfd8f50495eb4fb5f0ef0966bfd', 'request_id': 'req-49450c2b-d7f8-4b46-b463-69a787752a0e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.299374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] The token cache is being invalidate because project 1407bcfd8f50495eb4fb5f0ef0966bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:30.299664 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.300474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.302029 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1298/2589] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:30 2022] DELETE /identity/v3/projects/1407bcfd8f50495eb4fb5f0ef0966bfd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:30.306032 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:30.316471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fb69f-525b-4f36-83c3-815f54f029cc service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:30.328119 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:30.398492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:30.399144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:30.399430 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:30.399712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] PATH_INFO: `/v3/users/9e3f4cfbb43f49ccb50327a0f1c766b9` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:30.400892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3f4cfbb43f49ccb50327a0f1c766b9)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:30.402979 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:30.469141 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3f4cfbb43f49ccb50327a0f1c766b9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.481102 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3f4cfbb43f49ccb50327a0f1c766b9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.493821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3f4cfbb43f49ccb50327a0f1c766b9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.494517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3f4cfbb43f49ccb50327a0f1c766b9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.501989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29f6ac60-043c-471f-873b-742918e57356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3f4cfbb43f49ccb50327a0f1c766b9'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:30.508245 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1292/2590] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:30 2022] DELETE /identity/v3/users/9e3f4cfbb43f49ccb50327a0f1c766b9 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:30.866248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:30.874018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:30.900536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49450c2b-d7f8-4b46-b463-69a787752a0e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.011365 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.012322 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.012629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.012905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] PATH_INFO: `/v3/projects/bc42da86bd864d31aec70603c22a5775` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.014044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc42da86bd864d31aec70603c22a5775)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.015965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.058852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:31.072679 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc42da86bd864d31aec70603c22a5775', 'request_id': 'req-ed163418-68ec-4908-8256-a63b2f413374'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:31.082181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc42da86bd864d31aec70603c22a5775', 'request_id': 'req-ed163418-68ec-4908-8256-a63b2f413374'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:31.088712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc42da86bd864d31aec70603c22a5775', 'request_id': 'req-ed163418-68ec-4908-8256-a63b2f413374'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:31.101198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] The token cache is being invalidate because project bc42da86bd864d31aec70603c22a5775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:31.101486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:31.102168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ed163418-68ec-4908-8256-a63b2f413374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:31.108498 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1299/2591] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:30 2022] DELETE /identity/v3/projects/bc42da86bd864d31aec70603c22a5775 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:31.110192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.111214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.111589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.142800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:31.144052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:31.202446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1293/2592] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:31.210266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.210549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.210816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.230706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:31.231878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:31.240335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1300/2593] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:31.244431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.254057 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac3b978-2afa-4917-9ee2-869d1ddf539b None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.258864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.259460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.259743 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.260045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.260716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:31.261020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.262675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.277055 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1294/2594] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:31.280672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.291605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990777-9ce1-4004-9fa2-69cbc00c93a1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.297621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.298994 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.299268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.299676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.300397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.302602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.339168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1301/2595] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:31.342905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.361241 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bde698-8d0f-4f62-9034-61e3050a4c1d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.382097 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:31.424830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.425471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.425750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.426025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.426695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.428805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.449681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1295/2596] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:31.453889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.465514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea055c-35c5-431d-877e-ffb851a9cf18 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.470884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.471493 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.471763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.472078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.472760 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.474347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.487732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1302/2597] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:31.491655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.502164 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205bd730-5215-4ebc-831d-a5daea979fad admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.506782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.507382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.507657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.507924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.508594 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.510293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bf22b3-daaa-411c-b9e6-7c0625dd8539 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.523952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1296/2598] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:31.527680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.543568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb08aa-fcbf-419d-9573-049c659d26ad admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.549084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.549828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.550184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.550462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] PATH_INFO: `/v3/projects/7743d60f9eaf423095a56671709db20e/users/2a2f2dfe2e134c0fb3df3f3847dacd95/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.572065 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7743d60f9eaf423095a56671709db20e, user_id=2a2f2dfe2e134c0fb3df3f3847dacd95, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.574853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.591680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1303/2599] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:31 2022] PUT /identity/v3/projects/7743d60f9eaf423095a56671709db20e/users/2a2f2dfe2e134c0fb3df3f3847dacd95/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:31.604663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.604949 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.605292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.641256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] MFA Rules not processed for user `2a2f2dfe2e134c0fb3df3f3847dacd95`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:31.643113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:31.715930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1297/2600] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:31.721529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.789671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c356c15-bdca-4207-bf67-2b10e4081c10 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.794696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.795312 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.795595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.795874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.796543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.799123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.830497 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1304/2601] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:31.836798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.850935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa72eb-d017-4ed7-9a42-a994c4410553 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:31.944998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:31.945616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:31.945899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:31.946175 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:31.946813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:31.948886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:31.982646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1298/2602] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:31 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:31.983913 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:31.995961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae37f88-7014-45f2-9351-529524bbdeb9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.004018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.004821 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.005099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.005489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.006144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.008500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.025630 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1305/2603] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:32.029052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.044655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56f867-840a-4a33-ae68-dfe1a5eb7daf admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.051934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.053207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.053837 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.054529 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.055775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.058310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7a18c-ae03-42f1-8779-e363538031b7 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.073103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1299/2604] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:32.076938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.090467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdce1f-66d9-4024-987d-b39669112564 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.097453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.098863 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.099335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.100602 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] PATH_INFO: `/v3/projects/0d53c520cadf4800993dc81c7c4c976b/users/16a83408587d49cdbde1bb94c2fced25/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.121439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d53c520cadf4800993dc81c7c4c976b, user_id=16a83408587d49cdbde1bb94c2fced25, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.127378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.148388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1306/2605] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:32 2022] PUT /identity/v3/projects/0d53c520cadf4800993dc81c7c4c976b/users/16a83408587d49cdbde1bb94c2fced25/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:32.163868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.164150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.164478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.198470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:32.209013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] MFA Rules not processed for user `16a83408587d49cdbde1bb94c2fced25`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:32.210247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:32.282287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1300/2606] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:32.286697 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.296160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:32.392308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84284697-c4e0-42e7-a1a4-5b5ae27bc296 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.397533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.398262 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.398618 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.398956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.399830 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.401920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.434732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1307/2607] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:32 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:32.435916 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.448812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84349c73-eab3-49a2-8b5a-ed490f1a1ac7 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.510572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.511265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.511527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.511725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.512514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.514460 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.541933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1301/2608] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:32 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:32.543083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.554035 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208b2a1-44c7-4dfe-8c25-9096966cfaf2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.559290 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.559946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.560204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.560461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.561102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.562699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.577013 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1308/2609] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:32.578471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.589370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565d987-0e29-425e-b587-0e9720605cdc admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.594067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.595314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.595929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.596208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] PATH_INFO: `/v3/projects/9be24cf1c1b44383890291e78080b511/users/d46d06d923994627a89a05b6a1f5ab9c/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.612054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be24cf1c1b44383890291e78080b511, user_id=d46d06d923994627a89a05b6a1f5ab9c, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.614652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.627812 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1302/2610] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:32 2022] PUT /identity/v3/projects/9be24cf1c1b44383890291e78080b511/users/d46d06d923994627a89a05b6a1f5ab9c/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:32.632032 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.643131 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e32ba-351a-4edd-b857-9452ae0f31ca admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.705145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.705765 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.706042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.706391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.707244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.709289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630f6c4-68bb-4cfe-918d-0c1f29259d4e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.721649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1309/2611] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:32.725773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.736940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b901afe-f291-4c0a-9bd9-5a820d454001 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.741504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.742153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.742453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.742735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] PATH_INFO: `/v3/projects/9be24cf1c1b44383890291e78080b511/users/d46d06d923994627a89a05b6a1f5ab9c/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.744717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be24cf1c1b44383890291e78080b511, user_id=d46d06d923994627a89a05b6a1f5ab9c, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:32.747358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:32.758745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1303/2612] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:32 2022] PUT /identity/v3/projects/9be24cf1c1b44383890291e78080b511/users/d46d06d923994627a89a05b6a1f5ab9c/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:32.769174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.769457 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.769726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.800512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] MFA Rules not processed for user `d46d06d923994627a89a05b6a1f5ab9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:32.801703 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:32.889769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1310/2613] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:32.902154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:32.904004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:32.975410 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471d6c7-1a64-42c4-b95e-715d8bd197b2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:32.979608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:32.980359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:32.980656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:32.981080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:32.993967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:33.120745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:33.122795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:33.135088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1304/2614] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:34.812810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:34.815063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:34.822027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:34.828279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63760eb8-bd35-4ee5-a74a-b439a8f00d32 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:34.832676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:34.833275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:34.833546 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:34.833806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:34.901756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:34.903208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:34.913054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1311/2615] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:35.543024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:35.552585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6670025-2f69-4a3f-be77-6fea3785d2a7 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:35.556254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:35.556944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:35.557287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:35.557623 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:35.567059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:35.568586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0733b92f-771a-427f-8a3b-3720593a2f86 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:35.577830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1305/2616] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:36.967919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:36.973114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.066020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9c99-c196-489a-be2a-93adf6148822 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.072380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.073037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.073316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.073640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] PATH_INFO: `/v3/users/9af80c1235db4ebaaf30ba33c1e93d1b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.074770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9af80c1235db4ebaaf30ba33c1e93d1b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.076754 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:37.125730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9af80c1235db4ebaaf30ba33c1e93d1b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.132827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9af80c1235db4ebaaf30ba33c1e93d1b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.133444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9af80c1235db4ebaaf30ba33c1e93d1b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.140474 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9af80c1235db4ebaaf30ba33c1e93d1b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.152579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9af80c1235db4ebaaf30ba33c1e93d1b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.158889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1312/2617] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:36 2022] DELETE /identity/v3/users/9af80c1235db4ebaaf30ba33c1e93d1b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:37.163367 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.163829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.163946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.183209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:37.184446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.244236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1306/2618] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:37.254354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:37.255876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.278330 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:37.316154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee21ed9d-b3c8-4976-8c03-8e7ff4bfd557 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.321962 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.322566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.322844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.323117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.333178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.334631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226ff9a-e4d2-4495-94c3-4c9876023ea4 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:37.344226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1313/2619] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:37.618037 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:37.619511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.631784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33437369-aa83-4af0-a380-fa52c1b60ee2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.636573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.637206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.637478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.637740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] PATH_INFO: `/v3/projects/862082d6e756429ea5829677b8100dcc` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.639035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862082d6e756429ea5829677b8100dcc)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.640935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:37.686887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862082d6e756429ea5829677b8100dcc', 'request_id': 'req-49e62eb9-28b5-447e-b092-61082aa94497'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.694505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862082d6e756429ea5829677b8100dcc', 'request_id': 'req-49e62eb9-28b5-447e-b092-61082aa94497'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.700134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862082d6e756429ea5829677b8100dcc', 'request_id': 'req-49e62eb9-28b5-447e-b092-61082aa94497'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.711431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] The token cache is being invalidate because project 862082d6e756429ea5829677b8100dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:37.711717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.712349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-49e62eb9-28b5-447e-b092-61082aa94497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:37.713804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1307/2620] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:37 2022] DELETE /identity/v3/projects/862082d6e756429ea5829677b8100dcc => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:37.718694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.718977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.719245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.738302 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:37.739492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.797387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1314/2621] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:37.806445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.806757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.807036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.826826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:37.828019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:37.836414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1308/2622] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:37.840461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:37.849896 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104b255-a0d4-45b9-bb0e-13563373d210 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.854748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.855335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.855606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.855871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.856533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:37.856833 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.858450 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:37.870368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1315/2623] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:37.874572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:37.884159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01225c27-8030-4da3-bd59-79ead77bfb4e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.888762 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.889349 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.889626 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.889897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.890512 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.893043 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:37.898384 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:37.920727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1309/2624] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:37 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:37.924815 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:37.934565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee15500-987b-4908-a193-3700e05a94b9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:37.990633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:37.991261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:37.991473 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:37.991743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:37.992397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:37.994296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:38.015366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1316/2625] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:37 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:38.019921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:38.030248 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347bdc7e-7bbe-4f6f-8fd6-3f495c1702a5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:38.035516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:38.036148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:38.036465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:38.036745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:38.037381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:38.039272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:38.049619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1310/2626] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:38.053506 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:38.063559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca628f4b-0a96-4cea-9f08-e9f14862b8d9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:38.079244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:38.079244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:38.079244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:38.079244 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:38.079665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:38.079665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc35b71-f2b7-45e2-8375-28169a5e72b2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:38.086495 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1317/2627] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:38.088330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:38.099412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481df8e-cb84-4aed-a159-9e9b13e2d871 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:38.104381 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:38.105121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:38.105461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:38.105739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] PATH_INFO: `/v3/projects/e0f05f191ce347808feae0a0acfc0934/users/b3846bf16f804e9294b0e5bed556a233/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:38.122924 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f05f191ce347808feae0a0acfc0934, user_id=b3846bf16f804e9294b0e5bed556a233, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:38.125836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:38.160199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1311/2628] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:38 2022] PUT /identity/v3/projects/e0f05f191ce347808feae0a0acfc0934/users/b3846bf16f804e9294b0e5bed556a233/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:38.162052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:38.162739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:38.163019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:38.193390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] MFA Rules not processed for user `b3846bf16f804e9294b0e5bed556a233`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:38.194612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:38.213883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:38.285060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1318/2629] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:38.297257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:38.299219 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:38.367703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e76cf-f8f7-4c88-852a-2b527cf98c83 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:38.371325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:38.371915 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:38.372188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:38.372477 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:38.381185 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:38.382691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37a0a3-a899-4752-8e6a-f90f87098977 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:38.393450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1312/2630] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:39.531603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:39.534026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:39.599426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612745c0-80e5-4f36-bd2d-8968b9bcc08e None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:39.604243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:39.604886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:39.605159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:39.605427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] PATH_INFO: `/v3/users/2a2f2dfe2e134c0fb3df3f3847dacd95` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:39.606517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a2f2dfe2e134c0fb3df3f3847dacd95)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:39.608434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:39.655288 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a2f2dfe2e134c0fb3df3f3847dacd95'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:39.663447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a2f2dfe2e134c0fb3df3f3847dacd95'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:39.664080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a2f2dfe2e134c0fb3df3f3847dacd95'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:39.670998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a2f2dfe2e134c0fb3df3f3847dacd95'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:39.682625 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a2f2dfe2e134c0fb3df3f3847dacd95'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:39.688469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1319/2631] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:39 2022] DELETE /identity/v3/users/2a2f2dfe2e134c0fb3df3f3847dacd95 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:39.692907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:39.693369 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:39.693463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:39.712727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:39.713909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:39.773624 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1313/2632] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:39.783652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:39.785110 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:39.830323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:39.848593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ae0e1-39b0-41c7-8f92-cd5655329352 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:39.852289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:39.853118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:39.853388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:39.853653 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:39.862929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:39.864375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:39.874341 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1320/2633] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:40.158370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:40.160230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:40.174031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389a70c-749c-4233-aa44-c925fe02e372 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:40.178948 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:40.179676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:40.180050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:40.180568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] PATH_INFO: `/v3/projects/7743d60f9eaf423095a56671709db20e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:40.181754 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7743d60f9eaf423095a56671709db20e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:40.183905 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:40.238471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7743d60f9eaf423095a56671709db20e', 'request_id': 'req-2f7035fe-f63f-44e0-9d2a-2fb870388017'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.240228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:40.247311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7743d60f9eaf423095a56671709db20e', 'request_id': 'req-2f7035fe-f63f-44e0-9d2a-2fb870388017'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.255177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7743d60f9eaf423095a56671709db20e', 'request_id': 'req-2f7035fe-f63f-44e0-9d2a-2fb870388017'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.266415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] The token cache is being invalidate because project 7743d60f9eaf423095a56671709db20e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:40.266826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.268069 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.269603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1314/2634] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:40 2022] DELETE /identity/v3/projects/7743d60f9eaf423095a56671709db20e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:40.274127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:40.285161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4e1bf-f81c-4960-9a5f-7355dac4457e service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:40.343414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:40.344186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:40.344508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:40.344779 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] PATH_INFO: `/v3/users/16a83408587d49cdbde1bb94c2fced25` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:40.345923 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a83408587d49cdbde1bb94c2fced25)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:40.347828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:40.399871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a83408587d49cdbde1bb94c2fced25'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.406643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a83408587d49cdbde1bb94c2fced25'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.407509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a83408587d49cdbde1bb94c2fced25'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.414419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a83408587d49cdbde1bb94c2fced25'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.426643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a83408587d49cdbde1bb94c2fced25'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.432662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1321/2635] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:40 2022] DELETE /identity/v3/users/16a83408587d49cdbde1bb94c2fced25 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:40.721411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:40.722891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:40.739966 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7035fe-f63f-44e0-9d2a-2fb870388017 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:40.795937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:40.796596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:40.796874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:40.797142 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] PATH_INFO: `/v3/projects/0d53c520cadf4800993dc81c7c4c976b` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:40.798235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d53c520cadf4800993dc81c7c4c976b)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:40.800799 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:40.844985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d53c520cadf4800993dc81c7c4c976b', 'request_id': 'req-67fe578c-8414-41ce-8339-76b80b6c0f5c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.852937 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d53c520cadf4800993dc81c7c4c976b', 'request_id': 'req-67fe578c-8414-41ce-8339-76b80b6c0f5c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.858766 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d53c520cadf4800993dc81c7c4c976b', 'request_id': 'req-67fe578c-8414-41ce-8339-76b80b6c0f5c'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.870851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] The token cache is being invalidate because project 0d53c520cadf4800993dc81c7c4c976b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:40.871149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.871790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:40.873661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1315/2636] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:40 2022] DELETE /identity/v3/projects/0d53c520cadf4800993dc81c7c4c976b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:40.877551 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:40.878948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:40.891411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42706-e3a7-4e74-96c6-7bec54b8414c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:40.945978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d5030-b264-43de-93e4-f266637291db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:40.946611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5030-b264-43de-93e4-f266637291db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:40.946889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5030-b264-43de-93e4-f266637291db admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:40.947160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d5030-b264-43de-93e4-f266637291db admin admin] PATH_INFO: `/v3/users/d46d06d923994627a89a05b6a1f5ab9c` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:40.948258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5030-b264-43de-93e4-f266637291db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46d06d923994627a89a05b6a1f5ab9c)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:40.950184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d5030-b264-43de-93e4-f266637291db admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:40.951660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:40.999729 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-540d5030-b264-43de-93e4-f266637291db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46d06d923994627a89a05b6a1f5ab9c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.006858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-540d5030-b264-43de-93e4-f266637291db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46d06d923994627a89a05b6a1f5ab9c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.007488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-540d5030-b264-43de-93e4-f266637291db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46d06d923994627a89a05b6a1f5ab9c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.014301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-540d5030-b264-43de-93e4-f266637291db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46d06d923994627a89a05b6a1f5ab9c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.026414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-540d5030-b264-43de-93e4-f266637291db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46d06d923994627a89a05b6a1f5ab9c'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.032112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1322/2637] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:40 2022] DELETE /identity/v3/users/d46d06d923994627a89a05b6a1f5ab9c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:41.313541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.315297 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:41.337205 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fe578c-8414-41ce-8339-76b80b6c0f5c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.405135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.405769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.406048 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.406320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] PATH_INFO: `/v3/projects/9be24cf1c1b44383890291e78080b511` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.407425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be24cf1c1b44383890291e78080b511)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.409907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.422679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:41.464663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be24cf1c1b44383890291e78080b511', 'request_id': 'req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.473737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be24cf1c1b44383890291e78080b511', 'request_id': 'req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.481284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be24cf1c1b44383890291e78080b511', 'request_id': 'req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.493555 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] The token cache is being invalidate because project 9be24cf1c1b44383890291e78080b511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:41.493839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.494485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-10399b54-1a59-4c2e-80a7-ec8ff3ed335d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:41.496275 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1316/2638] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:41 2022] DELETE /identity/v3/projects/9be24cf1c1b44383890291e78080b511 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:41.500988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.501116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.501380 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.520997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:41.522209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:41.584276 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1323/2639] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:41.591985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.592263 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.592559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.613502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:41.614693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:41.622505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1317/2640] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:41.626745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.636251 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91121c-6e10-4f06-81ac-10bbe9d79136 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.641077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.641844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.642355 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.642613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.643360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:41.643667 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.645694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.657565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1324/2641] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:41.662313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.671302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c779cbe-6f63-4f49-bb67-a8d439c7ad52 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.676538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.677121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.677396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.677662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.678278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.680134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.706530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1318/2642] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:41 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:41.710547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.721028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd0c8-0b01-4646-b63d-480962ae48fe admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.780926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.781591 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.781944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.782230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.783527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.785837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.798680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:41.810216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1325/2643] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:41 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:41.814267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.824779 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598efbf7-3763-4edd-902e-32a1ed20d34e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.829385 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.830549 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.830823 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.831091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.831714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.833368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.843697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1319/2644] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:41.848852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.859135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbebd4cb-7b85-48d4-9185-12ed967d3a17 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.864053 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.864694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.864969 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.865235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.865864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.867463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622efb91-e772-4f3f-aafd-dc78af564f53 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.878565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1326/2645] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:41.882578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:41.893302 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8882406-cca7-42d4-b17f-208f9e2b428e admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:41.897922 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:41.898640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.898920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.899191 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] PATH_INFO: `/v3/projects/961cdcb79c4b468fb77a07d1af8af6bf/users/5ca704c1462943d1b5a4ce57e3743465/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.914073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] RBAC: Authorizing `identity:create_grant(project_id=961cdcb79c4b468fb77a07d1af8af6bf, user_id=5ca704c1462943d1b5a4ce57e3743465, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:41.916650 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:41.929350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1320/2646] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:41 2022] PUT /identity/v3/projects/961cdcb79c4b468fb77a07d1af8af6bf/users/5ca704c1462943d1b5a4ce57e3743465/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:41.940361 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467debb7-af8e-43f1-9664-7139f9896a48 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:41.940706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467debb7-af8e-43f1-9664-7139f9896a48 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:41.941006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467debb7-af8e-43f1-9664-7139f9896a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:41.968793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-467debb7-af8e-43f1-9664-7139f9896a48 None None] MFA Rules not processed for user `5ca704c1462943d1b5a4ce57e3743465`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:41.969975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-467debb7-af8e-43f1-9664-7139f9896a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:42.030772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1327/2647] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:42.042662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:42.044360 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:42.099194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8807ef-6320-4a3f-a875-cc840f661141 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:42.102996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:42.103563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:42.103839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:42.104106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:42.112149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:42.113607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:42.123731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1321/2648] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:43.674648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:43.675210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:43.675210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:43.698161 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:43.699351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:43.877993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1328/2649] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:36:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:43.891304 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:43.893433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:43.934310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:43.963393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f131bb-57fa-4526-b6c0-b486458bf814 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:43.968448 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:43.969051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:43.969330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:43.969604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:43.977838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:43.979418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936f6f96-b966-4722-be5e-c8155439091b service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:43.987458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1322/2650] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:36:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:36:44.715080 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:44.777463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de773487-c4cd-4cf7-a2d9-0909cae6a046 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:44.782567 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:44.783176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:44.783448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:44.783711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] PATH_INFO: `/v3/users/5ca704c1462943d1b5a4ce57e3743465` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:44.785007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca704c1462943d1b5a4ce57e3743465)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:44.786906 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:44.809205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:44.834990 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca704c1462943d1b5a4ce57e3743465'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:44.841579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca704c1462943d1b5a4ce57e3743465'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:44.842303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca704c1462943d1b5a4ce57e3743465'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:44.849394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca704c1462943d1b5a4ce57e3743465'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:44.861153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca704c1462943d1b5a4ce57e3743465'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:44.873132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1329/2651] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:44 2022] DELETE /identity/v3/users/5ca704c1462943d1b5a4ce57e3743465 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:44.873298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:44.873298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:44.873298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:44.892678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:44.893914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:44.950632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1323/2652] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:44.960354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:44.961763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:45.017427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb9054-cd02-456e-a65f-4fa0a949c9f2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.020999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.021592 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.021865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.022129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.030872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.032271 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b12194-b277-4833-9bf2-19866b65e35b service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.041619 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1330/2653] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:45.313298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.314894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:45.330669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85072ad3-1f05-4380-8896-c6e022fa941f None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.335852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.336739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.337019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.337405 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] PATH_INFO: `/v3/projects/961cdcb79c4b468fb77a07d1af8af6bf` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.338425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=961cdcb79c4b468fb77a07d1af8af6bf)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.340355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.391699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '961cdcb79c4b468fb77a07d1af8af6bf', 'request_id': 'req-0190a7b6-e32e-46db-b397-0f476d013256'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:45.407898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '961cdcb79c4b468fb77a07d1af8af6bf', 'request_id': 'req-0190a7b6-e32e-46db-b397-0f476d013256'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:45.415120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '961cdcb79c4b468fb77a07d1af8af6bf', 'request_id': 'req-0190a7b6-e32e-46db-b397-0f476d013256'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:45.427161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] The token cache is being invalidate because project 961cdcb79c4b468fb77a07d1af8af6bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:45.427544 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:45.428203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-0190a7b6-e32e-46db-b397-0f476d013256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:45.429732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1324/2654] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:45 2022] DELETE /identity/v3/projects/961cdcb79c4b468fb77a07d1af8af6bf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:45.435066 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.436007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.436408 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.456716 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:45.457943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:45.515446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1331/2655] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:45.523463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.523748 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.524024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.543945 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:45.545180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:45.553146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1325/2656] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:45.557242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.566848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a713a7-60d9-47ae-aedb-e639bd7372a1 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.571580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.572178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.572489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.572758 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.573390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:45.573685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.575282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.587206 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1332/2657] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:45.591273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.601025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe6f3-cfc3-47c1-b7b1-6a9e85795ff4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.605789 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.606401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.606673 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.606939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.607558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.609724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.635841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1326/2658] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:45 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:45.640036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.651285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87561-d169-4972-a4f5-248b087bfcf3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.678465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:45.717903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.718504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.718776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.719042 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.719674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.722477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.743545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1333/2659] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:45 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:45.747416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.758187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40674b04-6b68-4a30-9275-a3ea30b95290 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.762725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.763314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.763585 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.763850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.764655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.766571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.777201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1327/2660] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:45.781260 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.794882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ef41-f390-4ecb-a38d-65d95bb264db admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.799548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.800159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.800470 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.800739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.801368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.802938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc06486-b7b9-450f-b0f2-83252342d78a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.812683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1334/2661] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:45.816889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:45.826738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ae58ce-8ef6-4e71-8d15-33c77060d7d4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:45.831998 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:45.832664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.832944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.833214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] PATH_INFO: `/v3/projects/10e36126bfe84d91bff01576b4b31996/users/c816e4baaaf64d8888bdb1ed6e541927/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.841487 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:45.849808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10e36126bfe84d91bff01576b4b31996, user_id=c816e4baaaf64d8888bdb1ed6e541927, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:45.852851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:45.867061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1328/2662] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:45 2022] PUT /identity/v3/projects/10e36126bfe84d91bff01576b4b31996/users/c816e4baaaf64d8888bdb1ed6e541927/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:45.934079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:45.934375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:45.934674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:45.964128 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] MFA Rules not processed for user `c816e4baaaf64d8888bdb1ed6e541927`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:45.965414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:46.029980 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1335/2663] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:46.041786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:46.043560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:46.100352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b75e4e-9d85-43e8-aff4-5ee4b07aa6c5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:46.104180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:46.104795 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:46.105067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:46.105334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:46.113373 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:46.114773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952bc1c-9723-4b81-9790-fbf8be773b14 service glance] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:46.123796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1329/2664] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:46.824488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:46.825877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:46.895025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24954e-b028-4dfa-b65a-eec0c34ea00b None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:46.899859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:46.900525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:46.900802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:46.901070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] PATH_INFO: `/v3/users/b3846bf16f804e9294b0e5bed556a233` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:46.902210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3846bf16f804e9294b0e5bed556a233)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:46.904117 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:46.946168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:46.953299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3846bf16f804e9294b0e5bed556a233'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:46.961033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3846bf16f804e9294b0e5bed556a233'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:46.961652 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3846bf16f804e9294b0e5bed556a233'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:46.969282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3846bf16f804e9294b0e5bed556a233'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:46.982187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3846bf16f804e9294b0e5bed556a233'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:46.989199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1336/2665] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:46 2022] DELETE /identity/v3/users/b3846bf16f804e9294b0e5bed556a233 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:46.994114 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:46.994306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:46.994445 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.015093 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:47.016343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:47.078281 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1330/2666] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:47.088806 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.091982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:47.155270 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8873685-5ad8-4ab2-8823-042f2dde1f52 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.158909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.160075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.160377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.160651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.169538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.171036 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11284c8-2e7e-4da3-975a-a9b87c6dda59 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.180976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1337/2667] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:36:47.466222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.467687 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:47.482199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022b0ec-04d0-436a-8820-cb42a20d809d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.487382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.488335 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.488674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.489030 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] PATH_INFO: `/v3/projects/e0f05f191ce347808feae0a0acfc0934` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.490552 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f05f191ce347808feae0a0acfc0934)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.493094 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.541930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f05f191ce347808feae0a0acfc0934', 'request_id': 'req-f86581a6-aef4-4059-a966-222fcedcbdeb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:47.549841 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f05f191ce347808feae0a0acfc0934', 'request_id': 'req-f86581a6-aef4-4059-a966-222fcedcbdeb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:47.555718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f05f191ce347808feae0a0acfc0934', 'request_id': 'req-f86581a6-aef4-4059-a966-222fcedcbdeb'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:47.567315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] The token cache is being invalidate because project e0f05f191ce347808feae0a0acfc0934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:47.567603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:47.568250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-f86581a6-aef4-4059-a966-222fcedcbdeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:47.569845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1331/2668] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:47 2022] DELETE /identity/v3/projects/e0f05f191ce347808feae0a0acfc0934 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:47.574604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.574883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.575152 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.595683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:47.596917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:47.661468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1338/2669] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:47.670681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.670962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.671234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.691657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:47.692935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:47.701274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1332/2670] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:47.705482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.716005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b700348-ae1c-477c-877c-a9f465d8cf6c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.720735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.721326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.721635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.721905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.722533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:47.722832 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.724452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.736364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1339/2671] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:47.741253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.750325 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f25a0f-e91a-4770-8c7d-10ea818b343d None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.755617 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.756202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.756504 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.756772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.757393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.759261 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.763607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:47.788559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1333/2672] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:47 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:47.792880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.802998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca8c86-1fcb-45c8-882a-d8aef654282f admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.845175 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:47.860261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.860912 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.861178 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.861446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.862088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.864539 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.886973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1340/2673] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:47 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:47.891090 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.901194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb77a8-a872-4ce6-a2a3-ce4cc7e0200f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.908709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.909332 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.909608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.909901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.910546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.912128 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.922536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1334/2674] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:47.927118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.936948 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b7073-22b6-487c-bf0d-4f81d3c5aaf3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.942127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.942711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.942988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.943253 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.943878 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.945484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686df98-162e-4f3d-be4c-9e38791727b8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:47.956828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1341/2675] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:47.961031 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:47.971004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb8724-3182-48d5-881e-8c5c046c9028 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:47.975516 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:47.976159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:47.976471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:47.976745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] PATH_INFO: `/v3/projects/b4bfcff1436641c9a9d348826ea3c234/users/ad07392d04a642cf986dbb19a2c99b0d/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:47.991621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4bfcff1436641c9a9d348826ea3c234, user_id=ad07392d04a642cf986dbb19a2c99b0d, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:47.994291 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.009198 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1335/2676] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:47 2022] PUT /identity/v3/projects/b4bfcff1436641c9a9d348826ea3c234/users/ad07392d04a642cf986dbb19a2c99b0d/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:48.019459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.019804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.020159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.047133 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] MFA Rules not processed for user `ad07392d04a642cf986dbb19a2c99b0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:48.048360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:48.112748 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1342/2677] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:48.117256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.179059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a285683b-0e59-43a3-b018-5c816bee8714 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.183665 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.184260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.184563 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.184835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.185446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.187361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.214022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1336/2678] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:48 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:48.218071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.229594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c618b44-5af9-454c-aa16-6618cf30dc2a None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.284937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.285565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.285995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.286153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.286670 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.288587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.310984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1343/2679] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:48 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:48.315024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.325577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026787ba-f6c0-446e-8e54-a39d8f9f6aef admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.330226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.330836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.331110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.331377 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.332010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.333613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.344442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1337/2680] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:48.349144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.359309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a626946-00e4-4fb1-828e-80a8ffd888c0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.364631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.365215 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.365491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.365900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] PATH_INFO: `/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d/users/982ffe520beb48508b58989e1f6a4ff8/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.381310 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0196a9ac7bc47d38a44d8d6994f067d, user_id=982ffe520beb48508b58989e1f6a4ff8, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.383954 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.396600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1344/2681] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:48 2022] PUT /identity/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d/users/982ffe520beb48508b58989e1f6a4ff8/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:48.400606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.410829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbf1b02-52d3-4ebd-9c0a-fee00d998d7b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.468614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.469226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.469501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.469769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.470400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.472509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef301f6a-89fc-4494-8ef8-7a83b851906e admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.482781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1338/2682] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:48.487040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.497846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879626b-33de-4f86-965f-e94d756865db admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.502565 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.503204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.503481 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.503756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] PATH_INFO: `/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d/users/982ffe520beb48508b58989e1f6a4ff8/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.505683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0196a9ac7bc47d38a44d8d6994f067d, user_id=982ffe520beb48508b58989e1f6a4ff8, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.508327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.519514 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1345/2683] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:48 2022] PUT /identity/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d/users/982ffe520beb48508b58989e1f6a4ff8/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:48.529900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.530267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.530546 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.559783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] MFA Rules not processed for user `982ffe520beb48508b58989e1f6a4ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:48.561104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:48.569471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:48.631909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1339/2684] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:48.643205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.645672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:48.655864 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:48.705782 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1529b-6ce5-4f1d-9ec7-0b5a0aba1256 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.709965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.710562 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.710836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.711107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.720390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.722444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.732221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1346/2685] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:48.840109 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:48.850645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c1c72-05f2-4cd5-9783-52ccda697415 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:48.854591 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:48.855172 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:48.855412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:48.855732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:48.913928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:48.915404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a00f48-15ea-402f-81e6-f4e5e1008b0c service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:48.924061 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1340/2686] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:51.518552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:51.579353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f18759d-baff-4aae-8e41-e581baf2ea61 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:51.583999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:51.584646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:51.584922 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:51.585189 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] PATH_INFO: `/v3/users/ad07392d04a642cf986dbb19a2c99b0d` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:51.586281 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad07392d04a642cf986dbb19a2c99b0d)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:51.588167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:51.636046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad07392d04a642cf986dbb19a2c99b0d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:51.643136 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad07392d04a642cf986dbb19a2c99b0d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:51.643756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad07392d04a642cf986dbb19a2c99b0d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:51.650573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad07392d04a642cf986dbb19a2c99b0d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:51.662374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad07392d04a642cf986dbb19a2c99b0d'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:51.668606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1347/2687] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:51 2022] DELETE /identity/v3/users/ad07392d04a642cf986dbb19a2c99b0d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:51.673292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:51.673784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:51.673892 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:51.693872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:51.695066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:51.753233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1341/2688] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:51.763654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:51.765233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:51.796777 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:51.825882 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e1006-f151-4e48-a2b5-5aaefc25fc48 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:51.829531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:51.830203 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:51.830564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:51.830895 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:51.839778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:51.841207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:51.850849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1348/2689] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:52.142357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:52.143787 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:52.155719 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a8678-c5f1-4563-a951-7ea06c28efb3 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:52.162013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:52.162860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:52.163268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:52.163667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] PATH_INFO: `/v3/projects/b4bfcff1436641c9a9d348826ea3c234` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:52.167688 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4bfcff1436641c9a9d348826ea3c234)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:52.169747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:52.210734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:52.219826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4bfcff1436641c9a9d348826ea3c234', 'request_id': 'req-287a4901-9088-4754-8e58-0108ff9a8d5a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.228840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4bfcff1436641c9a9d348826ea3c234', 'request_id': 'req-287a4901-9088-4754-8e58-0108ff9a8d5a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.234627 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4bfcff1436641c9a9d348826ea3c234', 'request_id': 'req-287a4901-9088-4754-8e58-0108ff9a8d5a'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.245505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] The token cache is being invalidate because project b4bfcff1436641c9a9d348826ea3c234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:52.245788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.246407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.247893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1342/2690] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:52 2022] DELETE /identity/v3/projects/b4bfcff1436641c9a9d348826ea3c234 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:52.252031 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:52.261600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb4e7-1603-4184-8620-2673c530f573 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:52.331873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:52.333144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:52.333415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:52.333676 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] PATH_INFO: `/v3/users/982ffe520beb48508b58989e1f6a4ff8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:52.334820 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] RBAC: Authorizing `identity:delete_user(user_id=982ffe520beb48508b58989e1f6a4ff8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:52.336730 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:52.382366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982ffe520beb48508b58989e1f6a4ff8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.389443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982ffe520beb48508b58989e1f6a4ff8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.390156 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982ffe520beb48508b58989e1f6a4ff8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.401957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982ffe520beb48508b58989e1f6a4ff8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.414155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-dc7bdaad-f0e8-4887-8d5f-8f4d8ec640df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982ffe520beb48508b58989e1f6a4ff8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.419983 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1349/2691] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:52 2022] DELETE /identity/v3/users/982ffe520beb48508b58989e1f6a4ff8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:52.698310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:52.699830 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:52.714737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a4901-9088-4754-8e58-0108ff9a8d5a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:52.776661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:52.777301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:52.777581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:52.777859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] PATH_INFO: `/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:52.778964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0196a9ac7bc47d38a44d8d6994f067d)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:52.780930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:52.828113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0196a9ac7bc47d38a44d8d6994f067d', 'request_id': 'req-bb6becef-36f6-4777-b8d7-eae5db995258'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.835668 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0196a9ac7bc47d38a44d8d6994f067d', 'request_id': 'req-bb6becef-36f6-4777-b8d7-eae5db995258'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.844622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0196a9ac7bc47d38a44d8d6994f067d', 'request_id': 'req-bb6becef-36f6-4777-b8d7-eae5db995258'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.865930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] The token cache is being invalidate because project a0196a9ac7bc47d38a44d8d6994f067d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:52.866236 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.867190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-bb6becef-36f6-4777-b8d7-eae5db995258 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:52.872438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1343/2692] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:52 2022] DELETE /identity/v3/projects/a0196a9ac7bc47d38a44d8d6994f067d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:52.875368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:52.875823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:52.876427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:52.896776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:52.897993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:52.942201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:52.958019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1350/2693] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:52.967233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:52.967517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:52.967791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:52.990091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:52.991313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:52.999343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1344/2694] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:53.003557 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.013145 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94759d37-9197-46e9-a848-c787ffc4fd3d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.018003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.018610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.018880 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.019143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.019775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:53.020606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.022187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.038247 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1351/2695] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:53.042724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.051950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fcb3dd-120b-48a6-a9c0-4b95fcfd7542 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.056835 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.057452 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.057730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.058000 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.058646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.060801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.092331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1345/2696] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:53.096527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.107345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e60cf-5219-44b5-af31-ea00a15d3112 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.164526 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.165052 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.165325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.165594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.166207 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.168122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.189641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1352/2697] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:53.194049 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.206644 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461eeb79-efa1-4abe-8f43-025d9a60002c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.211804 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.212439 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.212717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.212986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.213616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.215200 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.225931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1346/2698] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:53.230147 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.240201 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1822407-3985-4fc5-9558-28dbef611320 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.244800 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.245383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.245655 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.245921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.246542 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.248106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140002d-62f1-484b-8d46-96798f8f66ca admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.258441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1353/2699] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:53.262664 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.273427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e5f7de-b0c0-433b-9f09-cd937fa83b3b admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.278056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.278705 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.278983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.279255 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] PATH_INFO: `/v3/projects/99fd8ee8e84e4ae2a4eccc059f5e09d8/users/b7c7b5b35b2e47b981ba0e34c1ced1c8/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.294653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fd8ee8e84e4ae2a4eccc059f5e09d8, user_id=b7c7b5b35b2e47b981ba0e34c1ced1c8, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.297409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.299967 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:53.314331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1347/2700] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:53 2022] PUT /identity/v3/projects/99fd8ee8e84e4ae2a4eccc059f5e09d8/users/b7c7b5b35b2e47b981ba0e34c1ced1c8/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:53.324621 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.324921 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.325188 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.356058 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] MFA Rules not processed for user `b7c7b5b35b2e47b981ba0e34c1ced1c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:53.357303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:53.421218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1354/2701] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:53.425643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.489438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd1ee6b-8d88-4e74-8b6b-2f29f90e31f6 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.494110 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.494703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.494982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.495256 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.495886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.497805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.525341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1348/2702] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:36:53.529274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.539965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1745eb0-18fc-4aaf-ad85-1ecdd8bd3ab9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.585533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:53.598957 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.599636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.599976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.600351 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.600979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.603219 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.624600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1355/2703] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:53.628709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.639640 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3a78-b096-4c8e-83a4-c66ed77ffe9d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.645929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.646519 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.646797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.647067 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.647692 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.649314 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.659775 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1349/2704] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:53.663847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.673798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e63718-2af4-4265-9ee4-fe8a4e051c93 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.678437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.679100 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.679374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.679638 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] PATH_INFO: `/v3/projects/48e36bd951144940b3d07d3e06a2b254/users/9823d57cd4344f1481edf4457e81e1b2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.695094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e36bd951144940b3d07d3e06a2b254, user_id=9823d57cd4344f1481edf4457e81e1b2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.697741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.711503 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1356/2705] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:53 2022] PUT /identity/v3/projects/48e36bd951144940b3d07d3e06a2b254/users/9823d57cd4344f1481edf4457e81e1b2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:53.715536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.725621 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f88c6-cb04-4346-afc8-06d537117e6d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.782562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.783154 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.783431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.783700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.784343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.785944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc39a5-c4da-445a-8ecd-d51a87b17444 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.795557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1350/2706] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:53.799740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.810723 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60485e76-45cd-4161-8e37-e94e1d858555 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:53.815417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:53.816061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.816376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.816651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] PATH_INFO: `/v3/projects/48e36bd951144940b3d07d3e06a2b254/users/9823d57cd4344f1481edf4457e81e1b2/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.819158 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48e36bd951144940b3d07d3e06a2b254, user_id=9823d57cd4344f1481edf4457e81e1b2, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:53.821839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:53.831872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1357/2707] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:53 2022] PUT /identity/v3/projects/48e36bd951144940b3d07d3e06a2b254/users/9823d57cd4344f1481edf4457e81e1b2/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:53.843129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:53.843411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:53.843708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:53.883028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] MFA Rules not processed for user `9823d57cd4344f1481edf4457e81e1b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:53.884250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:53.951087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:53.970309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1351/2708] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:53.981884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:53.983647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:54.041839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa76f69-11a8-480c-97d6-14c5c20537d7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:54.045634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:54.046238 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:54.046518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:54.046784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:54.056387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:54.057809 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:54.067549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1358/2709] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:54.157550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:54.167989 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52114a-7964-4df0-ad53-39d860d66fdc None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:54.171914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:54.172733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:54.173118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:54.173489 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:54.238015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:54.240126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5fc27-07fc-4f89-a733-b798cb994436 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:54.249092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1352/2710] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:54.522486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:54.585632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edf163-4c8d-4bbf-a5ab-df842c17de4d service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:54.590289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:54.590915 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:54.591190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:54.591458 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] PATH_INFO: `/v3/users/b7c7b5b35b2e47b981ba0e34c1ced1c8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:54.592599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c7b5b35b2e47b981ba0e34c1ced1c8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:54.594512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:54.638231 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:54.641094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c7b5b35b2e47b981ba0e34c1ced1c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:54.651740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c7b5b35b2e47b981ba0e34c1ced1c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:54.652411 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c7b5b35b2e47b981ba0e34c1ced1c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:54.659739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c7b5b35b2e47b981ba0e34c1ced1c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:54.672041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c7b5b35b2e47b981ba0e34c1ced1c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:54.678513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1359/2711] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:54 2022] DELETE /identity/v3/users/b7c7b5b35b2e47b981ba0e34c1ced1c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:54.683908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:54.684187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:54.684494 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:54.702965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:54.704531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:54.764802 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1353/2712] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:36:54.775086 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:54.776496 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:54.834091 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b760-076c-46ff-9f2e-35db48896a7b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:54.837745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:54.838339 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:54.838613 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:54.838879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:54.847500 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:54.849547 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:54.859394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1360/2713] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:55.150313 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:55.151772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:55.164781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de2f0a-f420-4de6-bf43-686369e46433 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:55.169571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:55.170260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:55.170561 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:55.170859 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] PATH_INFO: `/v3/projects/99fd8ee8e84e4ae2a4eccc059f5e09d8` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:55.171964 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fd8ee8e84e4ae2a4eccc059f5e09d8)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:55.173946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:55.221433 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fd8ee8e84e4ae2a4eccc059f5e09d8', 'request_id': 'req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.229269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fd8ee8e84e4ae2a4eccc059f5e09d8', 'request_id': 'req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.235253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fd8ee8e84e4ae2a4eccc059f5e09d8', 'request_id': 'req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.247606 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] The token cache is being invalidate because project 99fd8ee8e84e4ae2a4eccc059f5e09d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:55.247891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.248599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.250068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1354/2714] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:55 2022] DELETE /identity/v3/projects/99fd8ee8e84e4ae2a4eccc059f5e09d8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:55.254123 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:55.264388 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be5aef-dfef-4eef-b86d-aa9cde498d23 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:55.324229 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:55.324887 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:55.325157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:55.325419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] PATH_INFO: `/v3/users/9823d57cd4344f1481edf4457e81e1b2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:55.326517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9823d57cd4344f1481edf4457e81e1b2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:55.328428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:55.375491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9823d57cd4344f1481edf4457e81e1b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.383442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9823d57cd4344f1481edf4457e81e1b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.384047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9823d57cd4344f1481edf4457e81e1b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.390840 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9823d57cd4344f1481edf4457e81e1b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.392649 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:55.405688 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-abfbef9a-80f7-45be-a64f-669694b013c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9823d57cd4344f1481edf4457e81e1b2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.412289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1361/2715] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:36:55 2022] DELETE /identity/v3/users/9823d57cd4344f1481edf4457e81e1b2 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:55.686232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:55.687678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:55.708662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6b497d-3d31-42f9-8851-ac2a4b164ee3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:55.717025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:55.768913 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:55.769571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:55.769809 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:55.770086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] PATH_INFO: `/v3/projects/48e36bd951144940b3d07d3e06a2b254` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:55.771292 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48e36bd951144940b3d07d3e06a2b254)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:55.773315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:55.820564 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48e36bd951144940b3d07d3e06a2b254', 'request_id': 'req-15265b53-0660-4ec0-9429-ba5ee6031616'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.828003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48e36bd951144940b3d07d3e06a2b254', 'request_id': 'req-15265b53-0660-4ec0-9429-ba5ee6031616'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.833840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48e36bd951144940b3d07d3e06a2b254', 'request_id': 'req-15265b53-0660-4ec0-9429-ba5ee6031616'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.847044 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] The token cache is being invalidate because project 48e36bd951144940b3d07d3e06a2b254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:36:55.847328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.848548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-15265b53-0660-4ec0-9429-ba5ee6031616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:36:55.850014 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1355/2716] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:36:55 2022] DELETE /identity/v3/projects/48e36bd951144940b3d07d3e06a2b254 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:55.855044 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:55.855328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:55.855598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:55.876088 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:55.877328 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:55.944255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1362/2717] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:55.951947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:55.952235 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:55.952547 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:55.974472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:55.975677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:55.983768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1356/2718] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:55.988007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:55.998659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f27991-c1a1-481f-a0f7-04b1d2347365 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.003172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.003775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.004046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.004345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.005073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:36:56.005395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.007111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.018982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1363/2719] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:36:56.023783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.032917 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49058ec0-d305-4150-88d9-870201a4aa8c None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.038084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.038672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.038950 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.039257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.039833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.041722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.071011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1357/2720] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:56.075059 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.085415 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b6464-9f5b-4701-9a06-0ecd254444db admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.146525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.147057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.147326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.147588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.148216 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.150146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.172082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1364/2721] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:56.176318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.187162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b3574-7953-4b1a-a5cd-5b8de62a0ade admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.191889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.192531 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.192814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.193084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.193716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.195312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.205378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1358/2722] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:56.209953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.220477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a746b8ee-4845-479d-b0c2-af89b677dd0d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.226074 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.226685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.226959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.227226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.228112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.229778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef04da-3e90-4460-9baf-4c6df41daf9d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.240871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1365/2723] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:56.245010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.255042 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc224a6-22f9-4494-952f-f0ee28d86acd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.260149 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.261003 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.261285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.261558 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] PATH_INFO: `/v3/projects/fbcd06307c02433ab6b1b97381dddac2/users/85f69db5dd7845a4857a998ac0220122/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.277066 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbcd06307c02433ab6b1b97381dddac2, user_id=85f69db5dd7845a4857a998ac0220122, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.279690 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.294666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1359/2724] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:56 2022] PUT /identity/v3/projects/fbcd06307c02433ab6b1b97381dddac2/users/85f69db5dd7845a4857a998ac0220122/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:56.305037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.305482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.305594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.334468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] MFA Rules not processed for user `85f69db5dd7845a4857a998ac0220122`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:56.335674 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:56.375499 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:56.404789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1366/2725] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:56.409190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.450884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:56.472946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c18b1-1bd6-42c4-baed-0327818ca7ab admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.477887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.478486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.478768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.479051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.479683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.481615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.509793 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1360/2726] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:56.513982 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.525043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d0da7-07f1-49d2-ba7d-596f2cdf49c9 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.579852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.580579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.580851 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.581107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.581727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.584196 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.606186 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1367/2727] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:56.610232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.620858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f90638-35a4-477d-a5f1-478cfb864928 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.625635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.626227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.626507 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.626778 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.627411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.629036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.639382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1361/2728] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:56.643704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.654376 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a263c2-b4f9-432a-83a6-4391c6f736d9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.658910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.659429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.659699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.659961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.660640 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.662234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2dcc13-ffb1-4401-b994-9445c4cd379b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.671718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1368/2729] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:56.675648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.685941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37149962-0a16-4d46-83e5-aed35b913225 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.690670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.691226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.691523 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.692312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] PATH_INFO: `/v3/projects/2accea1db28c49079732196a8f74b7a0/users/574fe6e0db304fccb567863b176ec1b0/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.706715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2accea1db28c49079732196a8f74b7a0, user_id=574fe6e0db304fccb567863b176ec1b0, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.710058 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.722822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1362/2730] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:56 2022] PUT /identity/v3/projects/2accea1db28c49079732196a8f74b7a0/users/574fe6e0db304fccb567863b176ec1b0/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:36:56.734167 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.734440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.734705 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.762726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] MFA Rules not processed for user `574fe6e0db304fccb567863b176ec1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:56.763901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:56.828734 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1369/2731] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:56.833279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.897625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368ed0-6a2a-455a-bb74-3484be1208eb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.902247 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:56.902840 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:56.903117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:56.903393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:56.904004 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:56.905914 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:56.931679 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1363/2732] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:56.935769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:56.946934 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf7b3c5-6f95-43c1-beef-d74bbd557d8c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:56.990336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:57.003552 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.004153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.005007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.005274 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.005892 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.007780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.028788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1370/2733] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:36:56 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:36:57.032959 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:57.043267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520581b-8c83-4ff2-a81a-644b0c6fa857 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:57.048479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.049068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.049378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.049616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.050244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.052532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.064828 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1364/2734] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:36:57.069694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:57.082006 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a34c5-d9e8-444e-bec2-bb1b251f7473 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:57.091927 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.092708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.093051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.093383 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] PATH_INFO: `/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6/users/90478de2fd2b436381943a14f1583e33/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.112192 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2097511b9c4ecfbe62874fa33be7f6, user_id=90478de2fd2b436381943a14f1583e33, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.115422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.129998 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1371/2735] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:57 2022] PUT /identity/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6/users/90478de2fd2b436381943a14f1583e33/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:57.133694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:57.143557 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf66707-2a96-4cff-a8a9-985a6fe1a669 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:57.199755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.200393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.200674 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.200946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.201578 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.203145 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29719f-1dee-45ef-a590-3e5fc0d721e1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.205337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:36:57.214440 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1365/2736] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:36:57.218410 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:57.229213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd9d54-b793-42dc-9c8a-d8457a082ec9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:57.233698 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.234324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.234594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.234859 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] PATH_INFO: `/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6/users/90478de2fd2b436381943a14f1583e33/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.236741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2097511b9c4ecfbe62874fa33be7f6, user_id=90478de2fd2b436381943a14f1583e33, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.239939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.249960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1372/2737] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:36:57 2022] PUT /identity/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6/users/90478de2fd2b436381943a14f1583e33/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:36:57.260279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.260609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.260891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.288500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] MFA Rules not processed for user `90478de2fd2b436381943a14f1583e33`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:36:57.289700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:57.356538 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1366/2738] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:36:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:36:57.368641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:57.370379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:36:57.428171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfe71d-0c70-4e97-a5c6-3e6abf405998 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:57.431769 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:57.432392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:57.432668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:57.432931 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:57.489004 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:57.490343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:57.498838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1373/2739] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:58.990960 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:59.000388 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17fabb6-2137-4f61-a8d0-720fbaa01e14 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:59.003858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:59.005056 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:59.005331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:59.005600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:59.060576 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:59.061936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:59.071221 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1367/2740] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:36:59.575165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:36:59.583654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4416ca4-ff77-4fe4-b2a2-e5473fb92961 service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:36:59.587179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:36:59.587766 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:36:59.588041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:36:59.588326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:36:59.596901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:36:59.598865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52638314-c88b-4a6b-8665-38c62ff34a11 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:36:59.608191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1374/2741] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:36:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:03.463903 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:03.543601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:03.567203 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd0653-0308-4e22-b89f-64fb5eb32413 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:03.572614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:03.573491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:03.573776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:03.574045 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] PATH_INFO: `/v3/users/85f69db5dd7845a4857a998ac0220122` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:03.575178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85f69db5dd7845a4857a998ac0220122)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:03.577118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:03.628963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85f69db5dd7845a4857a998ac0220122'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:03.636722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85f69db5dd7845a4857a998ac0220122'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:03.650689 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85f69db5dd7845a4857a998ac0220122'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:03.651309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85f69db5dd7845a4857a998ac0220122'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:03.658437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85f69db5dd7845a4857a998ac0220122'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:03.664847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1368/2742] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:03 2022] DELETE /identity/v3/users/85f69db5dd7845a4857a998ac0220122 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:03.669540 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:03.670022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:03.670295 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:03.724277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:03.725519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:03.734541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:03.793212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1375/2743] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:03.803587 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:03.805022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:03.884734 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34b3b14-24eb-4f50-b60b-c2c1d3522240 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:03.888577 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:03.889148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:03.889493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:03.889792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:03.899717 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:03.901308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:03.911442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1369/2744] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:04.181632 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:04.183234 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:04.196812 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af107a31-f996-4c6b-91c1-6db91a0bd95d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:04.201513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:04.202232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:04.202578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:04.202918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] PATH_INFO: `/v3/projects/fbcd06307c02433ab6b1b97381dddac2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:04.204104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbcd06307c02433ab6b1b97381dddac2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:04.206115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:04.254294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbcd06307c02433ab6b1b97381dddac2', 'request_id': 'req-c05f4a50-c851-4a39-9417-50bad0c2b81e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.262108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbcd06307c02433ab6b1b97381dddac2', 'request_id': 'req-c05f4a50-c851-4a39-9417-50bad0c2b81e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.267940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbcd06307c02433ab6b1b97381dddac2', 'request_id': 'req-c05f4a50-c851-4a39-9417-50bad0c2b81e'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.279082 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] The token cache is being invalidate because project fbcd06307c02433ab6b1b97381dddac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:04.279368 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.280009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.281483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1376/2745] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:04 2022] DELETE /identity/v3/projects/fbcd06307c02433ab6b1b97381dddac2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:04.285524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:04.294957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29425115-b52b-4332-b346-7de053fd0c04 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:04.338250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:04.352253 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:04.353153 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:04.353429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:04.353700 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] PATH_INFO: `/v3/users/574fe6e0db304fccb567863b176ec1b0` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:04.354801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=574fe6e0db304fccb567863b176ec1b0)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:04.356745 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:04.400404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574fe6e0db304fccb567863b176ec1b0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.407459 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574fe6e0db304fccb567863b176ec1b0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.419210 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574fe6e0db304fccb567863b176ec1b0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.419839 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574fe6e0db304fccb567863b176ec1b0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.427121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574fe6e0db304fccb567863b176ec1b0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.432833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1370/2746] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:04 2022] DELETE /identity/v3/users/574fe6e0db304fccb567863b176ec1b0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:04.699775 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:04.701286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:04.714392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f4a50-c851-4a39-9417-50bad0c2b81e admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:04.777135 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:04.777759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:04.778037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:04.778308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] PATH_INFO: `/v3/projects/2accea1db28c49079732196a8f74b7a0` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:04.779400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2accea1db28c49079732196a8f74b7a0)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:04.781334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:04.819541 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:04.827671 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2accea1db28c49079732196a8f74b7a0', 'request_id': 'req-8ad5e59a-d00d-4741-9bd9-612398a91ca3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.835162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2accea1db28c49079732196a8f74b7a0', 'request_id': 'req-8ad5e59a-d00d-4741-9bd9-612398a91ca3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.840780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2accea1db28c49079732196a8f74b7a0', 'request_id': 'req-8ad5e59a-d00d-4741-9bd9-612398a91ca3'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.852060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] The token cache is being invalidate because project 2accea1db28c49079732196a8f74b7a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:04.852392 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.853268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.854709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1377/2747] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:04 2022] DELETE /identity/v3/projects/2accea1db28c49079732196a8f74b7a0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:04.858756 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:04.860183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:04.873786 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca66c4-4bd8-4b48-965c-393b20b71219 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:04.928539 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:04.929192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:04.929469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:04.929740 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] PATH_INFO: `/v3/users/90478de2fd2b436381943a14f1583e33` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:04.930872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90478de2fd2b436381943a14f1583e33)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:04.932801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:04.981718 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90478de2fd2b436381943a14f1583e33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.988777 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90478de2fd2b436381943a14f1583e33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:04.999050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:05.001758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90478de2fd2b436381943a14f1583e33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.002641 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90478de2fd2b436381943a14f1583e33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.011676 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-e67390bd-48a2-44e3-8fdc-b4b76f85d652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90478de2fd2b436381943a14f1583e33'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.017780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1371/2748] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:04 2022] DELETE /identity/v3/users/90478de2fd2b436381943a14f1583e33 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:05.313126 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.314610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:05.327861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5e59a-d00d-4741-9bd9-612398a91ca3 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.384846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.385471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.385746 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.386012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] PATH_INFO: `/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.387094 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2097511b9c4ecfbe62874fa33be7f6)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.389016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.435363 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2097511b9c4ecfbe62874fa33be7f6', 'request_id': 'req-0e186928-1810-4ceb-9928-a64a0735e027'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.443017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2097511b9c4ecfbe62874fa33be7f6', 'request_id': 'req-0e186928-1810-4ceb-9928-a64a0735e027'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.448558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2097511b9c4ecfbe62874fa33be7f6', 'request_id': 'req-0e186928-1810-4ceb-9928-a64a0735e027'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.459825 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] The token cache is being invalidate because project ce2097511b9c4ecfbe62874fa33be7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:05.460108 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.460752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-0e186928-1810-4ceb-9928-a64a0735e027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:05.462198 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1378/2749] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:05 2022] DELETE /identity/v3/projects/ce2097511b9c4ecfbe62874fa33be7f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:05.467193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.467476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.467742 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.516024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:05.517357 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:05.574295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1372/2750] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:05.582174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.582455 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.582737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.602413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:05.603847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:05.611468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1379/2751] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:05.615502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.624943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018a2bd-e79d-452c-b24b-c6cbe621a48a None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.629398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.630730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.631009 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.631284 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.631908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:05.632207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.633848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.646183 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1373/2752] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:37:05.650424 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.659548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b449ee-debe-429c-b78d-1dd222a9f026 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.664187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.664804 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.665084 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.665356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.666002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.667871 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.694827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1380/2753] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:05 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:05.698855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.710757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a4f5f-bf55-4565-bbd7-ce82d62b500a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.764878 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.765466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.765737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.766007 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.766672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.768535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.788857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1374/2754] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:05 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:05.792963 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.803595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ddb0f-d977-4fe3-8ca2-afb685b5b36a admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.808179 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.808823 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.809101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.809370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.810003 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.811582 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.821817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1381/2755] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:05.825662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.835359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26f80f-c99f-4a7e-bac6-8af2ad0dae93 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.839788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.840397 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.840672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.840939 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.841559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.843117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87681722-0cfd-46df-ae71-7be9767a56e1 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.853755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1375/2756] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:05.857773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:05.868467 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd41c2-4aec-44d8-8b37-981563cfde37 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:05.873194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:05.873838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.874116 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.874385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] PATH_INFO: `/v3/projects/a9d184fa546b4e38bd0567183a343f34/users/c8ee6fc112274b96912889307c4124e0/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.889680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d184fa546b4e38bd0567183a343f34, user_id=c8ee6fc112274b96912889307c4124e0, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:05.892375 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:05.894477 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:05.906658 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1382/2757] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:05 2022] PUT /identity/v3/projects/a9d184fa546b4e38bd0567183a343f34/users/c8ee6fc112274b96912889307c4124e0/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:05.970920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:05.971510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:05.971510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:05.999884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] MFA Rules not processed for user `c8ee6fc112274b96912889307c4124e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:06.001115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.014051 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:06.065595 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1376/2758] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:06.077504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:06.079166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.135096 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e754a-e5f2-4c92-91ab-7ee351c7b2ff admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:06.138694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:06.139294 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.139566 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.139829 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.148586 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:06.149996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331f0ad-6024-42d3-b017-edecb24e762d service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:06.158650 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1383/2759] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:06.184518 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:06.247954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b67499-0e79-4365-a14b-78fdb5a599bd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:06.252553 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:06.253208 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.253486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.253755 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] PATH_INFO: `/v3/users/c8ee6fc112274b96912889307c4124e0` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.254995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8ee6fc112274b96912889307c4124e0)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:06.256946 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:06.302994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8ee6fc112274b96912889307c4124e0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.310201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8ee6fc112274b96912889307c4124e0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.322035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8ee6fc112274b96912889307c4124e0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.322675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8ee6fc112274b96912889307c4124e0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.329581 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8ee6fc112274b96912889307c4124e0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.335600 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1377/2760] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:06 2022] DELETE /identity/v3/users/c8ee6fc112274b96912889307c4124e0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:06.340040 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.340358 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.340636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.359984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:06.361220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.420226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1384/2761] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:06.430573 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:06.431955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.471169 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:06.491658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31d259-3248-455a-82c0-047bfb4adeae admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:06.495246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:06.495831 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.496103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.496401 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.505015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:06.506417 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f137dd-520c-4a2c-b332-de037d67a686 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:06.515814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1378/2762] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:06.779659 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:06.781099 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.794919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72983c1-9cb0-48df-a7a0-b05b573ae477 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:06.799429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:06.800646 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.801001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.801447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] PATH_INFO: `/v3/projects/a9d184fa546b4e38bd0567183a343f34` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.802558 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d184fa546b4e38bd0567183a343f34)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:06.804463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:06.850530 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d184fa546b4e38bd0567183a343f34', 'request_id': 'req-d40bc289-b887-4289-8914-2ded65ce31cc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.858078 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d184fa546b4e38bd0567183a343f34', 'request_id': 'req-d40bc289-b887-4289-8914-2ded65ce31cc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.863802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d184fa546b4e38bd0567183a343f34', 'request_id': 'req-d40bc289-b887-4289-8914-2ded65ce31cc'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.875321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] The token cache is being invalidate because project a9d184fa546b4e38bd0567183a343f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:06.875607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.876240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-d40bc289-b887-4289-8914-2ded65ce31cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:06.877726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1385/2763] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:06 2022] DELETE /identity/v3/projects/a9d184fa546b4e38bd0567183a343f34 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:06.882996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.883278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.883548 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.902980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:06.904148 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.961258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1379/2764] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:06.968819 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:06.969106 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:06.969378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:06.988818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:06.990011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:06.997784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1386/2765] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:07.002158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.012088 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff06fc8-5793-4d43-88b1-a413a3c68344 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.016972 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f917d-01be-433e-a762-30483082150a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.017569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f917d-01be-433e-a762-30483082150a admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.017853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f917d-01be-433e-a762-30483082150a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.018129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f917d-01be-433e-a762-30483082150a admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.018771 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f917d-01be-433e-a762-30483082150a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:07.019092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f917d-01be-433e-a762-30483082150a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.021343 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f917d-01be-433e-a762-30483082150a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.032855 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1380/2766] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:37:07.036933 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.045999 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d08ff-f5de-405d-b452-e1616614f6ca None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.050579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.051166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.051442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.051709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.052354 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.054230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.065214 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:07.087414 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1387/2767] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:07 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:07.088492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f917d-01be-433e-a762-30483082150a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.098267 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f917d-01be-433e-a762-30483082150a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.155986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.156543 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.156826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.157099 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.157729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.159636 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.183929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1381/2768] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:07 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:07.184950 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.195405 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7cb77-c952-49fb-aa5c-443cb54dd883 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.199914 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.200524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.200798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.201061 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.201678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.203239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.213442 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1388/2769] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:07.217455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.227271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd3d1b-2f2c-4052-b838-3f2a7c74ac10 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.232125 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.232749 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.233028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.233301 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.233928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.235521 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f62242-7059-4f94-8fde-cba37e4fb692 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.246607 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1382/2770] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:07.250443 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.260726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57597f6-88ce-4fe3-a9fd-a6a97919fff8 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.265217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.265848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.266129 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.266419 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] PATH_INFO: `/v3/projects/4793b91c7212484c84960201b2d39114/users/1c6957aac61b4ed985e2dafc888c4b6a/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.281657 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4793b91c7212484c84960201b2d39114, user_id=1c6957aac61b4ed985e2dafc888c4b6a, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.284420 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.298761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1389/2771] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:07 2022] PUT /identity/v3/projects/4793b91c7212484c84960201b2d39114/users/1c6957aac61b4ed985e2dafc888c4b6a/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:07.304733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.305081 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.305418 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.324559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:07.325739 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:07.363469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:07.383815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1383/2772] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:07.393502 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:07.395185 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:07.453710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773bf2-ec3d-4c9a-adc8-092e9abc7ae9 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:07.457240 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:07.457827 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:07.458097 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:07.458360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:07.467016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:07.468484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9acfc0-b430-435d-90d9-f7191665b14e service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:07.477488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1390/2773] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:10.362438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:10.363254 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:10.478319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc79e4-bb30-4efc-ba3e-091fff325764 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:10.483957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:10.484953 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:10.485550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:10.486075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:10.488230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:10.502715 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:10.504724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:10.515579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1384/2774] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:11.506062 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.506580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.506580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.537531 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] MFA Rules not processed for user `1c6957aac61b4ed985e2dafc888c4b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:11.538835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:11.636051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1391/2775] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:11.640207 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:11.651485 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e39dc2-36a6-4cfe-96cf-0460171bb1b1 service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:11.656165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:11.656982 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.657293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.657567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.658204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:11.660115 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:11.689810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1385/2776] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:11 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:11.693706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:11.705072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1b9a9-9bce-4e3d-8cc9-da451bc4567d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:11.752258 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:11.765396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:11.766009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.766286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.766580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.767205 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:11.769305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:11.796162 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1392/2777] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:11 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:11.797464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:11.808188 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fcad59-e6cd-448b-85e6-6e12b24763a3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:11.814572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:11.815126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.815402 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.815671 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.816525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:11.817927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:11.829601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1386/2778] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:11.835736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:11.846483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb131bc0-6fba-4b39-aa86-f6d16be7b66b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:11.851876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:11.852807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.853093 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.853370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] PATH_INFO: `/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e/users/5f0141bfaf3e49a4989391525ebd2416/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.872604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5a48eb4ef14eb79bf6f09ed12ac93e, user_id=5f0141bfaf3e49a4989391525ebd2416, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:11.875235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:11.890909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1393/2779] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:11 2022] PUT /identity/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e/users/5f0141bfaf3e49a4989391525ebd2416/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:11.895608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:11.906589 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95afe3-e32c-41ef-b5e4-4a1de1031c71 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:11.977178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:11.978234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:11.978667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:11.979053 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:11.979845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:11.981682 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:11.992814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1387/2780] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:11.996985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:12.007533 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d419a1e-9132-4c4b-aea8-577914e7fd0d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:12.056518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:12.056518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:12.056518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:12.056518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] PATH_INFO: `/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e/users/5f0141bfaf3e49a4989391525ebd2416/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:12.059513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5a48eb4ef14eb79bf6f09ed12ac93e, user_id=5f0141bfaf3e49a4989391525ebd2416, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:12.062283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda65df-161a-48ba-9195-23703082b184 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:12.080235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1394/2781] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:11 2022] PUT /identity/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e/users/5f0141bfaf3e49a4989391525ebd2416/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:14.172569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:14.174345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:14.195491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f7b50-d95d-4bbc-a135-f28eaafa4919 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:14.209963 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:14.267035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:14.267769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:14.268249 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:14.268632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] PATH_INFO: `/v3/users/c816e4baaaf64d8888bdb1ed6e541927` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:14.269887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c816e4baaaf64d8888bdb1ed6e541927)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:14.272341 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:14.340695 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c816e4baaaf64d8888bdb1ed6e541927'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:14.355704 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c816e4baaaf64d8888bdb1ed6e541927'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:14.372328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c816e4baaaf64d8888bdb1ed6e541927'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:14.372980 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c816e4baaaf64d8888bdb1ed6e541927'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:14.382842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c816e4baaaf64d8888bdb1ed6e541927'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:14.397193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1388/2782] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:14 2022] DELETE /identity/v3/users/c816e4baaaf64d8888bdb1ed6e541927 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:14.398686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:14.399187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:14.399469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:14.424067 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:14.433118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:14.570968 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1395/2783] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:37:14.581437 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:14.584514 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:14.678696 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b21e8-42aa-4cc9-9f2a-c759aa2c2852 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:14.683193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:14.684011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:14.684423 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:14.684822 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:14.697075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:14.698556 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11214c8c-8b20-48cb-93d3-02448ab1b543 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:14.715920 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1389/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:15.343561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:15.346299 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:15.367025 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72450b-97de-462b-bdab-481ee2c929c6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:15.372920 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:15.373732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.384781 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.385072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] PATH_INFO: `/v3/projects/10e36126bfe84d91bff01576b4b31996` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.388977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10e36126bfe84d91bff01576b4b31996)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:15.391343 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:15.503242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10e36126bfe84d91bff01576b4b31996', 'request_id': 'req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:15.518735 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10e36126bfe84d91bff01576b4b31996', 'request_id': 'req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:15.529718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10e36126bfe84d91bff01576b4b31996', 'request_id': 'req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:15.545685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] The token cache is being invalidate because project 10e36126bfe84d91bff01576b4b31996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:15.546184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:15.547212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-b9e04a7f-f0a6-45c3-b356-5bc382a28ef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:15.548828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1396/2785] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:15 2022] DELETE /identity/v3/projects/10e36126bfe84d91bff01576b4b31996 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:15.553872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.554407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.554635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.590073 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:15.591711 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:15.647269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:15.694276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1390/2786] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:15.702509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.703724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.704072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.730142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:15.731371 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:15.743910 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1397/2787] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:15.745406 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:15.758333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605a3eb-ddfd-4403-9ac4-efed0aaefac0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:15.764268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:15.765189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.765805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.766409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.767308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:15.767738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:15.770295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:15.798502 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1391/2788] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:37:15.799805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:15.812190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32544c7e-556f-42da-9403-d4d5f8c0a6b6 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:15.817593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:15.818213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.818561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.818752 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.819391 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:15.821548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:15.843909 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:15.866952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1398/2789] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:15 2022] POST /identity/v3/projects => generated 397 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:15.868237 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:15.883218 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ce861-07b1-4501-91c4-a1327463afdb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:15.955165 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:15.956020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:15.956376 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:15.956662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:15.958106 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:15.960478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:15.988244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1392/2790] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:15 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:15.992755 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:16.007818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e0e50-299d-47da-b825-8038c453a5e7 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:16.029071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:16.029709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:16.029979 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:16.030246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:16.031636 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:16.033264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:16.053166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1399/2791] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:16.056712 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:16.076226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7fe90-9ad5-4f27-a88b-9ea0b09de03c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:16.089334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:16.089788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:16.090064 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:16.090334 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:16.091134 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:16.092605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266167d3-e25e-46c1-baf5-12cb8ac3bd0d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:16.118699 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1393/2792] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:16.122468 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:16.134860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc74303-038f-4a22-9ccf-f96e5bbdb2cc admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:16.140246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:16.141019 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:16.141373 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:16.141712 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] PATH_INFO: `/v3/projects/11611cc520244e75b27a027a44da0b70/users/7f25f580c835495ea5170cce738c8236/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:16.161916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11611cc520244e75b27a027a44da0b70, user_id=7f25f580c835495ea5170cce738c8236, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:16.166633 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:16.188060 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1400/2793] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:16 2022] PUT /identity/v3/projects/11611cc520244e75b27a027a44da0b70/users/7f25f580c835495ea5170cce738c8236/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:16.199213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64298e-1de7-4a87-852a-31427e7c50b2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:16.199584 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64298e-1de7-4a87-852a-31427e7c50b2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:16.199955 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e64298e-1de7-4a87-852a-31427e7c50b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:16.259052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9e64298e-1de7-4a87-852a-31427e7c50b2 None None] MFA Rules not processed for user `7f25f580c835495ea5170cce738c8236`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:16.261806 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9e64298e-1de7-4a87-852a-31427e7c50b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:16.392898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1394/2794] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:37:16.406104 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:16.408043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:16.486824 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e4f9da-1953-4685-a65e-ee1708030097 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:16.492102 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:16.493048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:16.493559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:16.494054 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:16.506886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:16.508601 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:16.528475 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1401/2795] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:16 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:16.999473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d275-eef1-4f2f-aa7f-2175ce5f9470 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:17.000084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d275-eef1-4f2f-aa7f-2175ce5f9470 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:17.000888 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42d275-eef1-4f2f-aa7f-2175ce5f9470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:17.037132 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-0a42d275-eef1-4f2f-aa7f-2175ce5f9470 None None] MFA Rules not processed for user `5f0141bfaf3e49a4989391525ebd2416`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:17.038646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-0a42d275-eef1-4f2f-aa7f-2175ce5f9470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:17.075020 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:17.150630 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1395/2796] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:17.166908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:17.169087 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:17.229736 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f9d48-4517-4162-a9b9-32371749d7ba service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:17.233357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '628750c611c24e8b9b4c0e66d2e3282e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:17.234022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:17.234360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:17.234710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:17.236594 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:17.297945 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:17.299360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:17.308166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1402/2797] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:17.450386 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:17.450678 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:17.450957 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:17.483300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:17.484551 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:17.555651 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1396/2798] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:37:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:17.578400 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:17.579841 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:17.600369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:17.659919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1f41b-6243-4deb-b3b8-c4724b5fdfab service glance] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:17.671654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:17.671654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:17.671654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:17.671654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:17.687217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:17.689603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e7b6d-f5a3-4d6b-8518-fa3286642ebf service swift] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:17.699146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1403/2799] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:37:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:37:23.991721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:24.073853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5bc99-a2f4-4405-9a8b-f96e1604a415 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:24.079213 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:24.079940 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:24.080458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:24.080746 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] PATH_INFO: `/v3/users/7f25f580c835495ea5170cce738c8236` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:24.081956 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f25f580c835495ea5170cce738c8236)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:24.083952 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:24.157579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f25f580c835495ea5170cce738c8236'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:24.162638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:24.168613 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f25f580c835495ea5170cce738c8236'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:24.187836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f25f580c835495ea5170cce738c8236'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:24.189117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f25f580c835495ea5170cce738c8236'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:24.197068 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f25f580c835495ea5170cce738c8236'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:24.212333 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1397/2800] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:23 2022] DELETE /identity/v3/users/7f25f580c835495ea5170cce738c8236 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:24.214336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:24.214771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:24.214943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:24.246436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:24.248917 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:24.345903 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1404/2801] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:24.363023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:24.365287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:24.496438 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e3e061-806e-4969-afff-9ee009859952 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:24.500672 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:24.501328 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:24.501603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:24.501874 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:24.511927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:24.513554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b448b-8c92-4957-84bb-17e8ba27dc8e service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:24.525757 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1398/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:24.874297 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:24.877472 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:24.895523 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf68467-069b-4aba-90b0-0facf8b4b47c None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:24.902308 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:24.903148 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:24.903427 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:24.903700 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] PATH_INFO: `/v3/projects/11611cc520244e75b27a027a44da0b70` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:24.905323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11611cc520244e75b27a027a44da0b70)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:24.907607 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:24.988808 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11611cc520244e75b27a027a44da0b70', 'request_id': 'req-054d108b-cd73-4a01-ab1e-db160c4ee8e7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:25.004852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11611cc520244e75b27a027a44da0b70', 'request_id': 'req-054d108b-cd73-4a01-ab1e-db160c4ee8e7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:25.012763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11611cc520244e75b27a027a44da0b70', 'request_id': 'req-054d108b-cd73-4a01-ab1e-db160c4ee8e7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:25.027634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] The token cache is being invalidate because project 11611cc520244e75b27a027a44da0b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:25.027953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:25.028807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-054d108b-cd73-4a01-ab1e-db160c4ee8e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:25.030604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1405/2803] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:24 2022] DELETE /identity/v3/projects/11611cc520244e75b27a027a44da0b70 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:26.831535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:26.832087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:26.832087 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:26.856968 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] MFA Rules not processed for user `1c6957aac61b4ed985e2dafc888c4b6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:26.858225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:26.921346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1399/2804] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:29.278485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:29.278997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:29.279305 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:29.300353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:29.301797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:29.328559 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:29.365708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1406/2805] 10.222.0.33 () {58 vars in 1009 bytes} [Thu Sep 22 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:29.377702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:29.380244 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:29.435330 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:29.451847 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1204197-715a-4dce-9a2e-937853b32975 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:29.456838 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:29.458011 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:29.458287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:29.458554 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:29.466436 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:29.468105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:29.476918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1400/2806] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:37:33.459696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:33.461248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:33.538444 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e476c7-5bad-47e2-89d8-1dc885edfa44 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:33.542538 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:33.543245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:33.543529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:33.543807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:33.546255 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:33.625695 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:33.628243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:33.640051 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1407/2807] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:37.507670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:37.509209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:37.524854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372fcd8-839b-45f4-a815-ef525367b335 service swift] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:37.529396 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:37.530010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:37.530288 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:37.530658 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] PATH_INFO: `/v3/users/262e5327ba144a6287fef6dd6293f886` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:37.531842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] RBAC: Authorizing `identity:delete_user(user_id=262e5327ba144a6287fef6dd6293f886)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:37.533797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:37.583808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262e5327ba144a6287fef6dd6293f886'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.590232 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262e5327ba144a6287fef6dd6293f886'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.601785 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262e5327ba144a6287fef6dd6293f886'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.602399 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262e5327ba144a6287fef6dd6293f886'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.609542 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7244637c-b4a2-4b1a-b217-974751a7276b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262e5327ba144a6287fef6dd6293f886'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.615331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1401/2808] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:37 2022] DELETE /identity/v3/users/262e5327ba144a6287fef6dd6293f886 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:37.788160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:37.799742 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc551-41f8-4e49-af84-52e917637bf4 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:37.863245 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:37.863884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:37.864160 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:37.864478 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] PATH_INFO: `/v3/projects/553395befb8f4c958d34b7dffb2317e7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:37.866210 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=553395befb8f4c958d34b7dffb2317e7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:37.868771 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:37.901786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:37.918785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553395befb8f4c958d34b7dffb2317e7', 'request_id': 'req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.926505 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553395befb8f4c958d34b7dffb2317e7', 'request_id': 'req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.932171 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553395befb8f4c958d34b7dffb2317e7', 'request_id': 'req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.944204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] The token cache is being invalidate because project 553395befb8f4c958d34b7dffb2317e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:37.944527 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.945166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fbd78ca5-f7fc-41eb-a0dd-4ba352d39acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:37.946857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1408/2809] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:37 2022] DELETE /identity/v3/projects/553395befb8f4c958d34b7dffb2317e7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:37.951566 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:37.951921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:37.952196 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:37.973545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:37.974811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:38.038891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1402/2810] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:38.047847 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.048448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.048791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.071118 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:38.072372 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:38.080377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1409/2811] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:38.084790 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.094848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f7ab2-1cfd-4b50-8f01-f98647893679 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.100025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.100813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.101199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.101567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.102285 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81376) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:38.102927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.104792 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.117781 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1403/2812] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 12:37:38.122266 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.132665 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4c8da-773b-42e7-bf84-a3b2f2c5eff7 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.137471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.138124 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.138399 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.138789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.139324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.141345 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.168041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1410/2813] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:38.172318 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.183052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b673c9e3-49a1-4161-b5e2-e9b9a5603bfd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.233050 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:38.250638 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.251270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.251545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.251810 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.252461 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.255217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.284893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1404/2814] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/users => generated 294 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:38.290451 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.309428 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f351-7629-4e85-8793-00d7a5a0bfd1 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.319811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.320641 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.321016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.321407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.322153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.323956 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.340138 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1411/2815] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:38.341725 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.358023 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3ad45-45a2-4979-b818-aa4cff57d864 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.363524 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.363983 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.364259 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.364567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.365190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.366780 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0af954-1a89-46db-865a-9acc62e99b7c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.379815 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1405/2816] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:38.383816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.394001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d59fa-4c1f-471a-bf96-f0b2f16908ab admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.399048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.399883 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.400564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.401028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] PATH_INFO: `/v3/projects/974df2cf44ec4d619299db0e93789db5/users/52ed7044d89942148276e0ed21cbaa5f/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.417876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974df2cf44ec4d619299db0e93789db5, user_id=52ed7044d89942148276e0ed21cbaa5f, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.420786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.435127 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1412/2817] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:38 2022] PUT /identity/v3/projects/974df2cf44ec4d619299db0e93789db5/users/52ed7044d89942148276e0ed21cbaa5f/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:38.445667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.446021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.446383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.480015 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] MFA Rules not processed for user `52ed7044d89942148276e0ed21cbaa5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:38.481464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:38.574646 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1406/2818] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:38.579027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.642014 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015146d4-4959-4b59-ab5f-e30348d61626 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.647015 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.647624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.647900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.648168 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.648828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.651828 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.685446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1413/2819] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:38.689488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.700428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7bf02-7494-43f3-a926-af99cd40adfd None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.758895 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.759776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.759776 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.760075 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.760681 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.762575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.785453 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1407/2820] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:38.789261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.799624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41b6b4-3f49-4bd0-bb8a-87c6377b500d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.804347 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.804967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.805241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.805509 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.806141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.807786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.818398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1414/2821] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:38.822490 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.838683 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adad1bf5-4267-4da5-95e1-bf737ee0f437 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.843269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.843863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.844137 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.844442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.845074 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.846886 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2b76f-0666-4e7c-bed5-5ec03669327d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.857933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1408/2822] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:37:38.866817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:38.877709 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14755f-0653-473c-8c69-5f671892a9fe admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:38.882564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:38.883243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.883520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.883788 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] PATH_INFO: `/v3/projects/2574bb7615b74d1fa8c28d1db64af0a5/users/7a6bb315020b4a948c6089e7345c61d5/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.900942 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2574bb7615b74d1fa8c28d1db64af0a5, user_id=7a6bb315020b4a948c6089e7345c61d5, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:38.903555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:38.918287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1415/2823] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:38 2022] PUT /identity/v3/projects/2574bb7615b74d1fa8c28d1db64af0a5/users/7a6bb315020b4a948c6089e7345c61d5/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:38.928629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:38.928881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:38.929157 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:38.957510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] MFA Rules not processed for user `7a6bb315020b4a948c6089e7345c61d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:38.958710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:38.979339 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:39.025645 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1409/2824] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:39.030083 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.035946 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:39.097323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f125c-178d-4654-a137-4947b816cf31 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.102045 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.102919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.103194 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.103463 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.104076 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.105997 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.132071 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1416/2825] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:39 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:39.136220 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.147464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c77c7c-4047-4fd6-852c-31395cba5142 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.202616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.203217 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.203496 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.203769 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] PATH_INFO: `/v3/users` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.204473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.206404 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.228182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1410/2826] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:39 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:39.232256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.242711 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6da9d-b0d1-475f-b6e1-c3beb6f970a0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.247390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.247986 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.248289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.248595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.249220 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.250799 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.261648 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1417/2827] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:39.265693 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.276722 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec957d-da19-49e1-9bce-e283122479cf admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.281286 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.281977 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.282258 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.282528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] PATH_INFO: `/v3/projects/396562a0c159433f87e82190f5ef4f10/users/50a9de5eb10b4023ab5041aa9b2579a7/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.296997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=396562a0c159433f87e82190f5ef4f10, user_id=50a9de5eb10b4023ab5041aa9b2579a7, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.299619 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.312500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1411/2828] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:39 2022] PUT /identity/v3/projects/396562a0c159433f87e82190f5ef4f10/users/50a9de5eb10b4023ab5041aa9b2579a7/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:39.316943 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.326504 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76897ad9-ba54-4585-91f0-026e430c5531 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.381056 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.381644 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.381916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.382181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.382796 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.384396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b3d61-f440-40db-981e-9f5deda868c8 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.393745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1418/2829] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:39.398409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.408493 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cf2f3-8d97-44fa-a54e-e875fd55d3dd admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.413034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.413670 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.413947 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.414214 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] PATH_INFO: `/v3/projects/396562a0c159433f87e82190f5ef4f10/users/50a9de5eb10b4023ab5041aa9b2579a7/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.416098 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=396562a0c159433f87e82190f5ef4f10, user_id=50a9de5eb10b4023ab5041aa9b2579a7, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.418782 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.428900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1412/2830] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:39 2022] PUT /identity/v3/projects/396562a0c159433f87e82190f5ef4f10/users/50a9de5eb10b4023ab5041aa9b2579a7/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:39.440120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.440446 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.440732 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.468673 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] MFA Rules not processed for user `50a9de5eb10b4023ab5041aa9b2579a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:39.469857 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:39.532039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:39.537789 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1419/2831] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:39.548316 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:39.549996 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:39.627346 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f4394-2475-4cf2-afd3-489db15360bb admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:39.631562 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:39.632150 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:39.632471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:39.632727 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:39.641455 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:39.642864 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:39.646710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:39.654962 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1413/2832] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:40.173872 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:40.175264 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:40.187959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ad3a22-df7f-40f6-a004-beb4747a9485 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:40.191708 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:40.192327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:40.192596 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:40.192858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:40.254265 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:40.256077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:40.265107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1420/2833] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:44.297171 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:44.307122 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a76f-a7ae-404a-96d0-6fcc97656a44 service cinder] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:44.310891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:44.311450 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:44.311729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:44.312001 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:44.385409 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:44.386891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace80a8f-1cdb-48f1-b2e4-b2361a1e1ff5 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:44.396708 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1414/2834] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:46.537660 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:46.599797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab43d05-8c18-4ec4-855c-6962d5f6282c service cinder] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:46.604604 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:46.605218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:46.605517 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:46.605780 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] PATH_INFO: `/v3/users/52ed7044d89942148276e0ed21cbaa5f` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:46.606867 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ed7044d89942148276e0ed21cbaa5f)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:46.608786 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:46.653518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ed7044d89942148276e0ed21cbaa5f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:46.662737 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ed7044d89942148276e0ed21cbaa5f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:46.663350 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ed7044d89942148276e0ed21cbaa5f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:46.670181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ed7044d89942148276e0ed21cbaa5f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:46.682163 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ed7044d89942148276e0ed21cbaa5f'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:46.687811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1421/2835] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:46 2022] DELETE /identity/v3/users/52ed7044d89942148276e0ed21cbaa5f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:46.692508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:46.692988 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:46.693086 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:46.712536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:46.713750 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:46.771753 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1415/2836] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:46.781398 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:46.782791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:46.801939 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:46.840662 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b9d25-66c6-4b41-adb2-c0c2eb65d595 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:46.844316 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:46.844908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:46.845181 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:46.845448 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:46.855041 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:46.856482 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:46.866635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1422/2837] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:47.150541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:47.152361 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:47.166121 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec7da-167f-499c-8a00-0412c86f8283 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:47.171981 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:47.172800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:47.173204 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:47.173575 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] PATH_INFO: `/v3/projects/974df2cf44ec4d619299db0e93789db5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:47.174791 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=974df2cf44ec4d619299db0e93789db5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:47.176850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:47.226505 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974df2cf44ec4d619299db0e93789db5', 'request_id': 'req-7c9aa70c-8ef0-4164-be81-c63a6164bd39'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.236133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974df2cf44ec4d619299db0e93789db5', 'request_id': 'req-7c9aa70c-8ef0-4164-be81-c63a6164bd39'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.242615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974df2cf44ec4d619299db0e93789db5', 'request_id': 'req-7c9aa70c-8ef0-4164-be81-c63a6164bd39'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.253709 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] The token cache is being invalidate because project 974df2cf44ec4d619299db0e93789db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:47.253994 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.254653 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.256274 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1416/2838] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:47 2022] DELETE /identity/v3/projects/974df2cf44ec4d619299db0e93789db5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:47.260209 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:47.271073 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8727dfa3-2368-4cb3-992c-9f5be568c6d5 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:47.329519 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:47.330249 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:47.330529 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:47.330849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] PATH_INFO: `/v3/users/7a6bb315020b4a948c6089e7345c61d5` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:47.332554 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a6bb315020b4a948c6089e7345c61d5)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:47.334430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:47.380268 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a6bb315020b4a948c6089e7345c61d5'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.387953 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a6bb315020b4a948c6089e7345c61d5'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.388669 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a6bb315020b4a948c6089e7345c61d5'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.397068 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a6bb315020b4a948c6089e7345c61d5'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.408900 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a6bb315020b4a948c6089e7345c61d5'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.414993 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1423/2839] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:47 2022] DELETE /identity/v3/users/7a6bb315020b4a948c6089e7345c61d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:47.703036 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:47.704633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:47.721072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9aa70c-8ef0-4164-be81-c63a6164bd39 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:47.747550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:47.790971 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:47.791596 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:47.791876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:47.792144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] PATH_INFO: `/v3/projects/2574bb7615b74d1fa8c28d1db64af0a5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:47.793251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2574bb7615b74d1fa8c28d1db64af0a5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:47.795130 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:47.843467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2574bb7615b74d1fa8c28d1db64af0a5', 'request_id': 'req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.851772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2574bb7615b74d1fa8c28d1db64af0a5', 'request_id': 'req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.859431 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2574bb7615b74d1fa8c28d1db64af0a5', 'request_id': 'req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.880532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] The token cache is being invalidate because project 2574bb7615b74d1fa8c28d1db64af0a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:47.880909 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.881597 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:47.883308 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1417/2840] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:47 2022] DELETE /identity/v3/projects/2574bb7615b74d1fa8c28d1db64af0a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:47.887303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:47.888767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:47.901773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a85c-e714-4099-a2c8-3027fe6af494 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:47.937440 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:47.973743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:47.973743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:47.973743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:47.973743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] PATH_INFO: `/v3/users/50a9de5eb10b4023ab5041aa9b2579a7` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:47.974230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a9de5eb10b4023ab5041aa9b2579a7)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:47.976465 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.031461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a9de5eb10b4023ab5041aa9b2579a7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.041595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a9de5eb10b4023ab5041aa9b2579a7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.042348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a9de5eb10b4023ab5041aa9b2579a7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.050817 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a9de5eb10b4023ab5041aa9b2579a7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.063629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-3b489aad-23ad-4633-ba60-c016e927a5dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a9de5eb10b4023ab5041aa9b2579a7'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.070261 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1424/2841] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:47 2022] DELETE /identity/v3/users/50a9de5eb10b4023ab5041aa9b2579a7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:48.339881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.341359 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:48.354720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d0aa9-83ef-4f1d-b969-18482e4e5b53 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.411294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.411918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.412197 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.412497 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] PATH_INFO: `/v3/projects/396562a0c159433f87e82190f5ef4f10` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.413599 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=396562a0c159433f87e82190f5ef4f10)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.415559 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.460479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '396562a0c159433f87e82190f5ef4f10', 'request_id': 'req-5c2ba12e-de45-4c26-9a74-3e520efceab0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.462890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:48.468579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '396562a0c159433f87e82190f5ef4f10', 'request_id': 'req-5c2ba12e-de45-4c26-9a74-3e520efceab0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.475738 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '396562a0c159433f87e82190f5ef4f10', 'request_id': 'req-5c2ba12e-de45-4c26-9a74-3e520efceab0'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.486721 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] The token cache is being invalidate because project 396562a0c159433f87e82190f5ef4f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:48.487002 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.487625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-5c2ba12e-de45-4c26-9a74-3e520efceab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:48.489143 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1418/2842] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:48 2022] DELETE /identity/v3/projects/396562a0c159433f87e82190f5ef4f10 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:48.495009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.495324 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.495598 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.514919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:48.516112 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:48.574190 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1425/2843] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:48.583201 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.583478 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.583744 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.603846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:48.605076 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:48.615176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1419/2844] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:48.619689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.629453 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0329f-595f-4e7e-b3b8-506a66c62339 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.634033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.634624 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.634899 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.635169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.635791 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:48.636090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.637714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.649589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1426/2845] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:37:48.653861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.663604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1fcef-a20e-440f-ab93-839f6f122627 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.668287 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.668842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.669120 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.669390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.670025 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.671944 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.697997 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1420/2846] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:48 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:48.702103 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.712232 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bac4f-7ed4-40f2-b769-5bd28797c381 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.752406 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:48.768603 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.769714 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.769988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.770254 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.770876 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.772785 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.794027 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1427/2847] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:48 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:48.798113 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.808158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f3116-6d6b-470d-bfda-2cb72f5ca4f2 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.812736 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.813319 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.813598 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.813868 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.814492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.816635 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.826276 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1421/2848] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:48.830617 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.842579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4d539-5771-4200-982f-ff27111c4a42 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.847199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.847793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.848070 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.848374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.849011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.850599 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b154d-d565-4653-aa8a-4a1c415e706f admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.862616 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1428/2849] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:48.866811 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:48.877272 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b7967-c89f-4bb9-8412-a113f4ded135 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:48.882428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:48.883101 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.883378 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.883649 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] PATH_INFO: `/v3/projects/279931a76843435eb214c700d16cc7b7/users/b8c221233d214b5eac2d424d61db94c8/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.898710 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=279931a76843435eb214c700d16cc7b7, user_id=b8c221233d214b5eac2d424d61db94c8, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:48.901380 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:48.915372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1422/2850] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:48 2022] PUT /identity/v3/projects/279931a76843435eb214c700d16cc7b7/users/b8c221233d214b5eac2d424d61db94c8/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:37:48.928184 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:48.928699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:48.928699 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:48.956072 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] MFA Rules not processed for user `b8c221233d214b5eac2d424d61db94c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:48.957835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.019571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1429/2851] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:49.031034 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:49.032730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.090629 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07db2b-6e2b-4579-86a4-56825e1acc60 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:49.094631 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:49.095230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.095509 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.095783 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.104610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:49.106072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90075488-3460-4a88-9c72-2f034a74d3d0 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:49.114902 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1423/2852] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:49.134548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:49.198919 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4befbd-e998-4997-a128-6b8a3eeec34d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:49.203418 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:49.204037 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.204353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.204595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] PATH_INFO: `/v3/users/b8c221233d214b5eac2d424d61db94c8` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.205715 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c221233d214b5eac2d424d61db94c8)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:49.208248 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:49.256174 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c221233d214b5eac2d424d61db94c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.264046 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c221233d214b5eac2d424d61db94c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.264682 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c221233d214b5eac2d424d61db94c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.265874 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:49.271967 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c221233d214b5eac2d424d61db94c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.286321 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c221233d214b5eac2d424d61db94c8'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.292462 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1430/2853] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:49 2022] DELETE /identity/v3/users/b8c221233d214b5eac2d424d61db94c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:49.296954 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.297238 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.297517 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.319193 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:49.320442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.385473 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1424/2854] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:49.395327 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:49.396743 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.457959 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45abe703-23e6-483b-97c0-648dc9dc44ad admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:49.461510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:49.462105 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.462377 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.462643 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.471770 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:49.473385 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33caa3-a246-4fd7-9c85-7d1f078aebce service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:49.483764 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1431/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:49.762569 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:49.764070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.776908 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8e6876-2c10-4a58-8d97-39d4d6471c4e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:49.781901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:49.782535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.782813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.783085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] PATH_INFO: `/v3/projects/279931a76843435eb214c700d16cc7b7` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.784192 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279931a76843435eb214c700d16cc7b7)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:49.786663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:49.803732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:49.833669 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279931a76843435eb214c700d16cc7b7', 'request_id': 'req-cadaf94a-3bc9-4866-97b9-63f3e4836818'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.840648 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279931a76843435eb214c700d16cc7b7', 'request_id': 'req-cadaf94a-3bc9-4866-97b9-63f3e4836818'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.846269 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279931a76843435eb214c700d16cc7b7', 'request_id': 'req-cadaf94a-3bc9-4866-97b9-63f3e4836818'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.857930 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] The token cache is being invalidate because project 279931a76843435eb214c700d16cc7b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:49.858222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.858861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-cadaf94a-3bc9-4866-97b9-63f3e4836818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:49.860370 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1425/2856] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:49 2022] DELETE /identity/v3/projects/279931a76843435eb214c700d16cc7b7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:49.866016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.866298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.866597 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.885835 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:49.887033 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.945960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1432/2857] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:49.954054 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:49.954398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:49.954615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:49.976126 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:49.977463 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:49.985991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1426/2858] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:49.990243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:49.999757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d674c-0da2-4ef2-9636-4e66adce2b30 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.004286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.004911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.005191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.005524 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.006159 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:50.006459 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.008157 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.026429 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1433/2859] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:37:50.027632 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.038536 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6c780-4b7f-4170-b339-94b44f5fa07e None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.043506 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.044133 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.044464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.044737 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.045362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.047250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.074059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1427/2860] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:50 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:50.078395 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.088839 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895394b-b721-4d6b-b040-026475c8e2af admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.143020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.143610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.143885 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.144154 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.145326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.147217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.155952 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:50.169629 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1434/2861] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:50 2022] POST /identity/v3/users => generated 279 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:37:50.173468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.183605 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842b31e-f78c-487d-bebf-4c215e524436 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.188080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.188697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.188969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.189234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.189857 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.191429 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.202019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1428/2862] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:50.205738 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.215973 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93244c3e-188f-406e-b4b7-1dc579842da1 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.220600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.221170 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.221452 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.221726 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.222356 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.224508 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cb467-4dee-42e8-9a02-32f71e1a4458 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.235043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1435/2863] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:50.238942 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.248798 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef243f-9f6c-47b0-b726-e30773d8190f admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.253383 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.254158 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.254532 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.254639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] PATH_INFO: `/v3/projects/a5c95824d97f4875999ce1b332ad0c6e/users/fed9447a571e4e5eabd0f8abe0e3468b/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.269022 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5c95824d97f4875999ce1b332ad0c6e, user_id=fed9447a571e4e5eabd0f8abe0e3468b, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.272170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.285571 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1429/2864] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:50 2022] PUT /identity/v3/projects/a5c95824d97f4875999ce1b332ad0c6e/users/fed9447a571e4e5eabd0f8abe0e3468b/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:50.295858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.296146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.296573 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.323696 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] MFA Rules not processed for user `fed9447a571e4e5eabd0f8abe0e3468b`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:50.324955 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:50.387813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1436/2865] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:50.422059 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.423763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:50.480052 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cfb3-8151-4881-983d-dc1cc8bf0a4c admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.483794 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.484427 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.484703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.484970 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.493350 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.494703 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae0a56-a58a-4490-a761-9ad255e7da8d service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.503467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1430/2866] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:50.519049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.581434 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a710451-e96e-46b4-8f92-db3597fb6e99 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.586635 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.587250 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.587525 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.587793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] PATH_INFO: `/v3/users/fed9447a571e4e5eabd0f8abe0e3468b` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.588911 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed9447a571e4e5eabd0f8abe0e3468b)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.590811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.636369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed9447a571e4e5eabd0f8abe0e3468b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:50.645122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed9447a571e4e5eabd0f8abe0e3468b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:50.645741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed9447a571e4e5eabd0f8abe0e3468b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:50.653007 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed9447a571e4e5eabd0f8abe0e3468b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:50.662370 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:50.668995 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed9447a571e4e5eabd0f8abe0e3468b'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:50.676317 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1437/2867] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:37:50 2022] DELETE /identity/v3/users/fed9447a571e4e5eabd0f8abe0e3468b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:50.680513 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.680796 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.681092 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.700271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:50.701475 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:50.738774 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:50.761355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1431/2868] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:50.771326 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:50.772731 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:50.829710 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bd62d5-15af-4cb8-be3c-89be83844d82 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:50.833413 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:50.834012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:50.834287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:50.834555 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:50.843275 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:50.844797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c86a96-09ca-4272-9ff6-b29349a14373 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:50.854285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1438/2869] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:51.127622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.129105 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:51.140797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81dbc4-8575-4e96-a3d3-caa6f29f1de0 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.145652 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.146362 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.146647 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.146901 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] PATH_INFO: `/v3/projects/a5c95824d97f4875999ce1b332ad0c6e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.148070 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5c95824d97f4875999ce1b332ad0c6e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.150161 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.196666 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5c95824d97f4875999ce1b332ad0c6e', 'request_id': 'req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:51.203877 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5c95824d97f4875999ce1b332ad0c6e', 'request_id': 'req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:51.209484 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5c95824d97f4875999ce1b332ad0c6e', 'request_id': 'req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:51.220421 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] The token cache is being invalidate because project a5c95824d97f4875999ce1b332ad0c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:37:51.220706 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:51.221358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-a13a09f1-4b6a-41d6-ac40-9ee8abca13fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:37:51.223356 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1432/2870] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:37:51 2022] DELETE /identity/v3/projects/a5c95824d97f4875999ce1b332ad0c6e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:37:51.229564 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.230039 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.230155 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.249759 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:51.250940 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:51.307447 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1439/2871] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:37:51.316260 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.316579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.316848 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.336225 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:51.337456 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:51.345251 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1433/2872] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:37:51.349772 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.359378 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0f8e2-cfcc-46ad-9108-4f68b0520b2d None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.364581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.365180 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.365437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.365704 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.366353 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:37:51.366647 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.368226 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.379306 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1440/2873] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:37:51.383306 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.392374 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9bafc-ba9b-490d-9e25-5c962a3094c4 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.396887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.397468 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.397747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.398019 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.398661 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.400568 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.429250 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1434/2874] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:37:51 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:51.433417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.443114 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22866fb-2db6-494a-b375-089795947ef2 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.497364 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.497951 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.498224 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.498486 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.499101 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.501020 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.521706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1441/2875] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:37:51 2022] POST /identity/v3/users => generated 283 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:37:51.525801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.536614 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88fe34-2920-4971-8f4d-a249b3cf0366 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.541180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.541767 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.542041 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.542310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.542933 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.544560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.554488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1435/2876] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:51.558256 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.567901 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d980dd1-87af-404a-b753-bc98c3776921 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.572433 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.573011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.573283 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.573549 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.574166 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.575725 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70680-66a1-4d10-ae92-61264954719d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.577724 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:37:51.590938 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1442/2877] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:37:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:37:51.592223 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.602824 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56dc35f-0f5c-47aa-94b8-ed88fefc1ce4 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.607394 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.608026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.608320 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.608604 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] PATH_INFO: `/v3/projects/27346ec2de2b47d7933bfb175d66f270/users/3205ec5db80a4d7f8be80fdd08b05ba2/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.622590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27346ec2de2b47d7933bfb175d66f270, user_id=3205ec5db80a4d7f8be80fdd08b05ba2, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.625233 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.638714 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1436/2878] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:37:51 2022] PUT /identity/v3/projects/27346ec2de2b47d7933bfb175d66f270/users/3205ec5db80a4d7f8be80fdd08b05ba2/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:37:51.650267 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.650548 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.650816 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.679654 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] MFA Rules not processed for user `3205ec5db80a4d7f8be80fdd08b05ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:37:51.680905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:51.743028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1443/2879] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:37:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 12:37:51.754129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:51.755797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:51.812189 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4ef88-f988-4b8f-8243-e6fdd284235d admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:51.815728 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:51.816869 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:51.817144 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:51.817411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:51.825442 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:51.826850 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50dac7f-d10b-44ee-a7d8-e6adf22a3865 service cinder] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:51.835934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1437/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:37:56.273491 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:37:56.274886 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:56.330992 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f5d11-f994-4ceb-a485-baa813f0f319 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:37:56.335212 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:37:56.335810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:37:56.336089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:37:56.336390 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:37:56.338115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:37:56.398560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:37:56.399908 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:37:56.409221 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1444/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:37:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:38:01.831527 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:38:06.405144 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=83 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:38:23.516691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101361-0bda-4cfa-a3c9-d2606c0525d2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:38:23.517167 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101361-0bda-4cfa-a3c9-d2606c0525d2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:38:23.517271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9101361-0bda-4cfa-a3c9-d2606c0525d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:38:23.536655 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-e9101361-0bda-4cfa-a3c9-d2606c0525d2 None None] MFA Rules not processed for user `f42af00458ac4046888a73397218f9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:38:23.538355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-e9101361-0bda-4cfa-a3c9-d2606c0525d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:38:23.591295 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1438/2882] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:38:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:38:27.232583 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:38:27.295728 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a1b62-4e7b-4438-926a-bb18b61e0002 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:38:27.300686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:38:27.301309 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:38:27.301581 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:38:27.301846 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] PATH_INFO: `/v3/users/3205ec5db80a4d7f8be80fdd08b05ba2` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:38:27.302937 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3205ec5db80a4d7f8be80fdd08b05ba2)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:38:27.305016 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:38:27.350988 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3205ec5db80a4d7f8be80fdd08b05ba2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:27.359340 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3205ec5db80a4d7f8be80fdd08b05ba2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:27.360030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3205ec5db80a4d7f8be80fdd08b05ba2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:27.367595 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3205ec5db80a4d7f8be80fdd08b05ba2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:27.379403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3205ec5db80a4d7f8be80fdd08b05ba2'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:27.386437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1445/2883] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:38:27 2022] DELETE /identity/v3/users/3205ec5db80a4d7f8be80fdd08b05ba2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:38:27.391628 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:38:27.392124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:38:27.392270 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:38:27.411135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:38:27.412358 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:38:27.470622 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1439/2884] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:38:27.480814 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:38:27.482208 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:38:27.537488 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a651e-fce9-4b8e-8135-fb6738f5dc31 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:38:27.541323 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:38:27.542241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:38:27.542241 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:38:27.542461 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:38:27.551773 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:38:27.553218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:38:27.563563 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1446/2885] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:38:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:38:28.054732 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:38:28.056273 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:38:28.070352 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee31095-364b-4f4e-abf0-fc5c497c7591 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:38:28.076166 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:38:28.076894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:38:28.077170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:38:28.077579 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] PATH_INFO: `/v3/projects/27346ec2de2b47d7933bfb175d66f270` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:38:28.078889 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27346ec2de2b47d7933bfb175d66f270)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:38:28.080969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:38:28.149784 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27346ec2de2b47d7933bfb175d66f270', 'request_id': 'req-fe67df9c-7959-4a72-83f1-b18e675583a3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:28.158601 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27346ec2de2b47d7933bfb175d66f270', 'request_id': 'req-fe67df9c-7959-4a72-83f1-b18e675583a3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:28.166080 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27346ec2de2b47d7933bfb175d66f270', 'request_id': 'req-fe67df9c-7959-4a72-83f1-b18e675583a3'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:28.180228 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] The token cache is being invalidate because project 27346ec2de2b47d7933bfb175d66f270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:38:28.180603 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:28.181483 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:38:28.183414 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1440/2886] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:38:28 2022] DELETE /identity/v3/projects/27346ec2de2b47d7933bfb175d66f270 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:38:37.559077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:38:38.176206 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:13.471578 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:13.472978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:13.529813 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319bf94-a715-4231-99f3-86bc01e61ae6 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:13.578522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e10e2b0b66754f128cddc94ca3daf101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:13.579115 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:13.579386 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:13.579651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:13.581439 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:13.660935 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:13.662366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b82c2-d432-4bb2-9c03-d5a8abde36f1 service cinder] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:13.684325 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1447/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:16.206379 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:16.207800 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:16.263713 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67df9c-7959-4a72-83f1-b18e675583a3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:16.267348 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:16.267943 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:16.268222 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:16.268525 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:16.270257 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:16.283894 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:16.285355 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:16.296834 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1441/2888] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:18.638336 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:18.638798 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:18.638893 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:18.687187 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] MFA Rules not processed for user `2fd208a28c9d41288c294cdf01bf6a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:18.688394 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:18.696620 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=100 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:18.750740 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1448/2889] 10.222.0.33 () {58 vars in 967 bytes} [Thu Sep 22 12:39:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:18.790567 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:18.791978 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:18.881300 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c6f1f-f8ad-4197-80c6-e9e58b2ffe91 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:18.886326 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '40dc194ca31947339986b511f4271498', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:18.886921 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:18.887199 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:18.887471 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:18.895364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:18.896975 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:18.904858 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1442/2890] 10.222.0.33 () {60 vars in 1384 bytes} [Thu Sep 22 12:39:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 12:39:25.347702 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:25.349315 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:25.423165 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac72e6f-20c7-418b-8801-8f3df2c5220f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:25.427902 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:25.428561 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:25.428838 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:25.429132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] PATH_INFO: `/v3/users/1c6957aac61b4ed985e2dafc888c4b6a` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:25.430282 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c6957aac61b4ed985e2dafc888c4b6a)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:25.432213 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:25.477803 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c6957aac61b4ed985e2dafc888c4b6a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.485235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c6957aac61b4ed985e2dafc888c4b6a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.485852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c6957aac61b4ed985e2dafc888c4b6a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.492960 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c6957aac61b4ed985e2dafc888c4b6a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.505287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c6957aac61b4ed985e2dafc888c4b6a'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.511741 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1449/2891] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:39:25 2022] DELETE /identity/v3/users/1c6957aac61b4ed985e2dafc888c4b6a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:25.683104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:25.694425 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb648b-262e-49f1-a706-f70901a06272 service swift] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:25.753118 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:25.753663 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:25.753936 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:25.754230 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] PATH_INFO: `/v3/projects/4793b91c7212484c84960201b2d39114` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:25.755311 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4793b91c7212484c84960201b2d39114)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:25.757227 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:25.772876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=100 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:25.801991 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4793b91c7212484c84960201b2d39114', 'request_id': 'req-2b352e41-eac6-42fd-b6c0-3df7663105e5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.809488 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4793b91c7212484c84960201b2d39114', 'request_id': 'req-2b352e41-eac6-42fd-b6c0-3df7663105e5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.815013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4793b91c7212484c84960201b2d39114', 'request_id': 'req-2b352e41-eac6-42fd-b6c0-3df7663105e5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.826177 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] The token cache is being invalidate because project 4793b91c7212484c84960201b2d39114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:39:25.826462 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.827084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.828582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1443/2892] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:39:25 2022] DELETE /identity/v3/projects/4793b91c7212484c84960201b2d39114 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:25.832631 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:25.834048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:25.847289 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfc626-e0a2-44a0-8a5a-c7a81a78762c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:25.873916 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:25.905242 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:25.905865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:25.906140 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:25.906409 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] PATH_INFO: `/v3/users/5f0141bfaf3e49a4989391525ebd2416` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:25.907520 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f0141bfaf3e49a4989391525ebd2416)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:25.909489 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:25.954706 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f0141bfaf3e49a4989391525ebd2416'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.961784 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f0141bfaf3e49a4989391525ebd2416'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.962426 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f0141bfaf3e49a4989391525ebd2416'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.969169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f0141bfaf3e49a4989391525ebd2416'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.981436 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-5aad8ed6-ea34-4aaa-8891-5bf4754e96e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f0141bfaf3e49a4989391525ebd2416'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:25.987049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1450/2893] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:39:25 2022] DELETE /identity/v3/users/5f0141bfaf3e49a4989391525ebd2416 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:26.154951 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.156393 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:26.169929 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b352e41-eac6-42fd-b6c0-3df7663105e5 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.225085 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.225731 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.226008 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.226279 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] PATH_INFO: `/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.228146 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5a48eb4ef14eb79bf6f09ed12ac93e)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.230104 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.275897 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5a48eb4ef14eb79bf6f09ed12ac93e', 'request_id': 'req-29c21560-b77c-4933-91f0-d477985d0537'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:26.282788 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5a48eb4ef14eb79bf6f09ed12ac93e', 'request_id': 'req-29c21560-b77c-4933-91f0-d477985d0537'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:26.288202 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5a48eb4ef14eb79bf6f09ed12ac93e', 'request_id': 'req-29c21560-b77c-4933-91f0-d477985d0537'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:26.299180 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] The token cache is being invalidate because project 6e5a48eb4ef14eb79bf6f09ed12ac93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:39:26.299458 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:26.300072 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-29c21560-b77c-4933-91f0-d477985d0537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:39:26.301535 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1444/2894] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:39:26 2022] DELETE /identity/v3/projects/6e5a48eb4ef14eb79bf6f09ed12ac93e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:26.307049 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.307334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.307606 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.327802 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:26.329048 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:26.390865 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1451/2895] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:26.398893 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.399178 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.399446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.423264 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:26.424851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:26.435821 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1445/2896] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:26.440090 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.450169 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847e3107-5941-4f87-a725-e607a98a94fe None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.454718 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.455300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.455571 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.455860 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.456534 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81377) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 12:39:26.456826 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.458396 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.470077 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1452/2897] 10.222.0.33 () {58 vars in 1153 bytes} [Thu Sep 22 12:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 12:39:26.474226 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.484162 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97e84b-6431-4f6e-8cef-ef0c016eb0c2 None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.488730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.489315 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.489620 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.489887 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.490508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.492412 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.516918 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:26.530773 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1446/2898] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:39:26 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:39:26.534844 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.545257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5cac7-271e-4932-a8ef-3c3a03c940a6 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.575672 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:26.601628 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.603417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.603685 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.603944 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.604575 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.606454 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.627043 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1453/2899] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:39:26 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 12:39:26.631117 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.641912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4b1fe-6a69-43fe-9103-a16c48501a90 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.646866 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.647446 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.647720 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.647986 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.648637 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.650941 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.662345 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1447/2900] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:39:26.666403 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.676348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c8fd9-7eff-4a7c-82b0-d01d5c6dc98d admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.680996 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.681587 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.681861 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.682130 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.682757 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.684393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfba536-89d5-401e-834b-4bd9222dbb5a admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.694612 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1454/2901] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:39:26.698464 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.709336 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb51cb-b00f-4e6e-b157-ebc56aace192 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.715770 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.716447 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.716730 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.717006 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] PATH_INFO: `/v3/projects/f4d530e1b20c47319f5e8949d808c1b3/users/7b3e6035de1e4fe3a651ece7d82842eb/roles/0568cdc72a2e49fb931bbf5cf5be87e5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.733398 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d530e1b20c47319f5e8949d808c1b3, user_id=7b3e6035de1e4fe3a651ece7d82842eb, role_id=0568cdc72a2e49fb931bbf5cf5be87e5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.736024 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.748833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1448/2902] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:39:26 2022] PUT /identity/v3/projects/f4d530e1b20c47319f5e8949d808c1b3/users/7b3e6035de1e4fe3a651ece7d82842eb/roles/0568cdc72a2e49fb931bbf5cf5be87e5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:26.753818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b9a3-d35e-4889-abd2-b3f42579bc58 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.754098 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b9a3-d35e-4889-abd2-b3f42579bc58 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.754366 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4b9a3-d35e-4889-abd2-b3f42579bc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.773183 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-c4e4b9a3-d35e-4889-abd2-b3f42579bc58 None None] MFA Rules not processed for user `9dcd38c10c5f47bf8d47fab51d035bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:26.774984 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-c4e4b9a3-d35e-4889-abd2-b3f42579bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:26.830588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1455/2903] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:26.841159 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:26.842860 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:26.901278 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258ef92-4cdd-49a7-b889-09e7ab23c0c3 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:26.904881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:26.905472 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:26.905747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:26.906021 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:26.914747 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:26.916176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:26.925400 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1449/2904] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:30.912437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:30.912985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:30.912985 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:30.943005 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] MFA Rules not processed for user `7b3e6035de1e4fe3a651ece7d82842eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:30.944367 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:31.011790 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1456/2905] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:31.016309 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.026856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b06e9-f21d-4ef7-91c0-e91ddc8bc8c0 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.031702 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.032416 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.032697 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.032969 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.033616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.035520 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.066390 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1450/2906] 10.222.0.33 () {60 vars in 1153 bytes} [Thu Sep 22 12:39:31 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:39:31.070510 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.081568 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a44d9-022e-458e-ba89-9ed4fdf48347 None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.128961 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:31.146853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.147483 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.147761 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.148028 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] PATH_INFO: `/v3/users` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.148905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.150877 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.178975 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1457/2907] 10.222.0.33 () {60 vars in 1144 bytes} [Thu Sep 22 12:39:31 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 12:39:31.184476 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.199265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd244b-a791-4597-a254-949d6c4588d9 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.206616 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.207268 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.207545 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.207813 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.208492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.210124 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.221833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1451/2908] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 12:39:31.226022 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.236848 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdc6e8-29d2-4904-93cf-7425ee056587 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.241374 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.242009 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.242286 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.242556 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] PATH_INFO: `/v3/projects/6ac63e237a6e4a008f318c662941f173/users/ed176c91eeee4a7aa8edd4cbb70a0514/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.263359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac63e237a6e4a008f318c662941f173, user_id=ed176c91eeee4a7aa8edd4cbb70a0514, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.266063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.280694 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1458/2909] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:39:31 2022] PUT /identity/v3/projects/6ac63e237a6e4a008f318c662941f173/users/ed176c91eeee4a7aa8edd4cbb70a0514/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:31.284643 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.295194 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de3073-6384-418e-aa24-f8331fc25547 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.354694 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.355211 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.355492 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.355763 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.356435 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.358017 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167458b-61e9-46ea-a811-4c70bf6b0b20 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.368096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1452/2910] 10.222.0.33 () {58 vars in 1122 bytes} [Thu Sep 22 12:39:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 12:39:31.372544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:31.383879 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e18916-ede9-49c4-80ce-a297f23f388c admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:31.388810 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:31.389469 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:31.389745 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:31.390017 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] PATH_INFO: `/v3/projects/6ac63e237a6e4a008f318c662941f173/users/ed176c91eeee4a7aa8edd4cbb70a0514/roles/97ed8ec57a984d71b7ba263e643b38fe` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:31.391905 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac63e237a6e4a008f318c662941f173, user_id=ed176c91eeee4a7aa8edd4cbb70a0514, role_id=97ed8ec57a984d71b7ba263e643b38fe)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:31.394611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:31.405125 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1459/2911] 10.222.0.33 () {60 vars in 1483 bytes} [Thu Sep 22 12:39:31 2022] PUT /identity/v3/projects/6ac63e237a6e4a008f318c662941f173/users/ed176c91eeee4a7aa8edd4cbb70a0514/roles/97ed8ec57a984d71b7ba263e643b38fe => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:39:35.919500 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500d80a-8357-4ab3-83ad-d3f1a31d09a1 None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:35.919987 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500d80a-8357-4ab3-83ad-d3f1a31d09a1 None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:35.920729 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500d80a-8357-4ab3-83ad-d3f1a31d09a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:35.962876 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-7500d80a-8357-4ab3-83ad-d3f1a31d09a1 None None] MFA Rules not processed for user `ed176c91eeee4a7aa8edd4cbb70a0514`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:35.964491 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-7500d80a-8357-4ab3-83ad-d3f1a31d09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:35.978899 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:36.050541 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1453/2912] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:36.070600 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:36.073407 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:36.161243 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69075b05-3acf-4169-875e-1fb68c0ccf79 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:36.168858 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:36.169579 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:36.169778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:36.170002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:36.231763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:36.233360 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:36.242929 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1460/2913] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:37.221415 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d535802-d055-407e-8787-67f2ca00fded None None] REQUEST_METHOD: `POST` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:37.221904 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d535802-d055-407e-8787-67f2ca00fded None None] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:37.222013 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d535802-d055-407e-8787-67f2ca00fded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:37.241337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.auth.core [None req-2d535802-d055-407e-8787-67f2ca00fded None None] MFA Rules not processed for user `7b3e6035de1e4fe3a651ece7d82842eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81376) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:39:37.242550 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2d535802-d055-407e-8787-67f2ca00fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:37.249413 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1454/2914] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:39:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:39:46.238235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:47.236608 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=25 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:39:52.914898 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:52.916382 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:52.970763 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915dad5-59aa-446c-babb-ba4079d0657e service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:52.974437 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:52.975030 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:52.975298 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:52.975560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:52.977513 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:52.987246 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:52.988690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:52.996422 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1461/2915] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:53.449863 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d535802-d055-407e-8787-67f2ca00fded None None] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:53.451898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2d535802-d055-407e-8787-67f2ca00fded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:53.520842 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d535802-d055-407e-8787-67f2ca00fded None None] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:53.525707 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:53.526298 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:53.526572 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:53.526836 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:53.529156 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:53.540084 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:53.541667 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:53.551234 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1455/2916] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:55.370296 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:55.380522 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985b0f9-a42b-461b-83e6-8305384d3fe1 service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:55.384176 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:55.384811 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:55.385085 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:55.385349 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:55.387089 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:55.397925 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:55.399397 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:55.409430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1462/2917] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:39:56.152854 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:39:56.154724 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:56.170312 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42741ba-a2fa-4139-b672-051a764658ed service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:39:56.176103 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:39:56.176814 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:39:56.177293 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:39:56.177660 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:39:56.179656 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:39:56.193510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:39:56.195091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:39:56.206872 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1456/2918] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:39:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:40:05.405873 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=32 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:06.202912 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=38 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:06.730918 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:40:06.744369 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd823a6-c4b6-46bf-9b73-8236de8e9dbd service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:40:06.749822 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:40:06.751026 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:40:06.751303 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:40:06.751589 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:40:06.753313 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:40:06.766235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:40:06.767852 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b408132-5148-4ec0-bc0d-e7bc5cb031e9 service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:40:06.778543 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1463/2919] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:40:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:40:08.998530 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:40:09.008353 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a25086-00c8-432d-8ce5-5c108f73d1dc service nova] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:40:09.011852 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:40:09.012479 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:40:09.012752 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:40:09.013016 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:40:09.022265 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:40:09.023662 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:40:09.033100 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1457/2920] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:40:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:40:16.774008 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:19.028768 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:28.407485 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] REQUEST_METHOD: `POST` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:40:28.408001 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:40:28.408113 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:40:28.427153 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.auth.core [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] MFA Rules not processed for user `f42af00458ac4046888a73397218f9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81377) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 12:40:28.428359 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:40:28.481683 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1464/2921] 10.222.0.33 () {58 vars in 958 bytes} [Thu Sep 22 12:40:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 12:40:28.807368 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:40:28.815856 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef608fd-4989-4e54-94b7-335d749149fa service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:40:28.819498 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:40:28.820091 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:40:28.820515 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:40:28.820797 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:40:28.822534 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:40:28.832691 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:40:28.834112 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:40:28.842176 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1458/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:40:38.474666 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=25 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:38.670079 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:40:38.683143 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c0040-7a6d-49f4-80fb-337e2503bd1f None None] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:40:38.688471 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:40:38.689057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:40:38.689329 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:40:38.689593 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:40:38.691279 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:40:38.701228 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:40:38.702801 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:40:38.711417 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1465/2923] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:40:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:40:38.838560 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:40:48.707476 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:10.346010 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:10.358136 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c955359-ae3f-43df-91af-c7448fcc3cc1 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:10.363026 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:10.363609 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:10.363883 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:10.364151 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:10.365881 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:10.379271 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:10.380938 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:10.391510 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1459/2924] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:15.445837 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:15.458218 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68056d-6512-42bd-a1a6-1e5170e4472f service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:15.464109 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6db6f737c3354ca692eb6ee91a496223', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:15.464739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:15.465012 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:15.465277 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:15.467230 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:15.480651 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:15.482204 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:15.493926 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1466/2925] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:20.386508 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=6 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:25.451168 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:25.461735 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239723-3e8a-4700-af07-940bdf0f2ae6 service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:25.465428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:25.466294 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:25.466829 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:25.467174 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:25.469129 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:25.479907 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:25.489680 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=6 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:25.490677 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:25.501896 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1460/2926] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:26.589018 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:26.599560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b08c57-e385-49be-99f8-48908cc0d50a service nova] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:26.603518 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:26.604111 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:26.604536 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:26.604691 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:26.606441 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:26.632202 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:26.633727 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:26.646122 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1467/2927] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:27.902625 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:27.911466 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cb18b1-8335-491a-9b88-8f4509d8cfc1 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:27.916805 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82a67430b66140e1b8427f2ffb627f42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:27.917610 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:27.918391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:27.918675 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:27.920931 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:27.996511 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:27.997934 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:28.008337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1461/2928] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:31.132002 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:31.144191 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665dc4a-7c85-46cc-b6d2-6395f3b9f81d service neutron] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:31.150492 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:31.151235 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:31.151611 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:31.152141 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:31.154301 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:31.168532 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:31.171767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:31.187689 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1468/2929] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:42:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:42:34.595985 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:34.598590 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:34.614135 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefe719-ad6f-4938-8040-a57e43ccdf91 service neutron] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:34.619716 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:34.626851 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:34.627407 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:34.629139 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] PATH_INFO: `/v3/users/f42af00458ac4046888a73397218f9f5` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:34.633127 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f42af00458ac4046888a73397218f9f5)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:34.635432 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:34.682898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f42af00458ac4046888a73397218f9f5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:34.689891 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f42af00458ac4046888a73397218f9f5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:34.701826 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f42af00458ac4046888a73397218f9f5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:34.702467 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f42af00458ac4046888a73397218f9f5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:34.710035 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f42af00458ac4046888a73397218f9f5'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:34.715928 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1462/2930] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:42:34 2022] DELETE /identity/v3/users/f42af00458ac4046888a73397218f9f5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:42:34.885776 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:34.897379 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc834803-2ce3-40a8-bb9c-0778ddc9fcaa service placement] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:34.952501 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:34.953120 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:34.954063 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:34.954352 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] PATH_INFO: `/v3/projects/5d44fc4fa85a4dfb82c31a602037a249` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:34.955430 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d44fc4fa85a4dfb82c31a602037a249)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:34.957348 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:35.001047 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d44fc4fa85a4dfb82c31a602037a249', 'request_id': 'req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.008217 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d44fc4fa85a4dfb82c31a602037a249', 'request_id': 'req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.013690 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d44fc4fa85a4dfb82c31a602037a249', 'request_id': 'req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.025884 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] The token cache is being invalidate because project 5d44fc4fa85a4dfb82c31a602037a249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:42:35.026172 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.026797 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.028239 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1469/2931] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:42:34 2022] DELETE /identity/v3/projects/5d44fc4fa85a4dfb82c31a602037a249 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:42:35.032457 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:35.033833 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:35.046898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8607872-c816-44a8-ae79-2f9190242204 admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:35.093428 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:35.103698 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:35.104331 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:35.104611 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:35.104884 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] PATH_INFO: `/v3/users/84eda3abf950456684a21245bbf17c55` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:35.105995 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84eda3abf950456684a21245bbf17c55)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:35.107911 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:35.152501 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84eda3abf950456684a21245bbf17c55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.159733 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84eda3abf950456684a21245bbf17c55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.171465 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84eda3abf950456684a21245bbf17c55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.172615 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84eda3abf950456684a21245bbf17c55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.179434 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84eda3abf950456684a21245bbf17c55'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.185382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1463/2932] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:42:35 2022] DELETE /identity/v3/users/84eda3abf950456684a21245bbf17c55 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:42:35.367793 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:42:35.369416 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:42:35.382978 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad6ec9-dff6-48ab-9535-dc2f5ba203de admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:42:35.439387 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:42:35.440011 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:42:35.440285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:42:35.440584 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] PATH_INFO: `/v3/projects/51980456e4f84b9183583477f03f3731` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:42:35.441663 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51980456e4f84b9183583477f03f3731)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:42:35.443544 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:42:35.449588 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:35.489739 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51980456e4f84b9183583477f03f3731', 'request_id': 'req-33c06ce7-b631-4849-b968-5b38cd8197d0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.496512 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51980456e4f84b9183583477f03f3731', 'request_id': 'req-33c06ce7-b631-4849-b968-5b38cd8197d0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.503075 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51980456e4f84b9183583477f03f3731', 'request_id': 'req-33c06ce7-b631-4849-b968-5b38cd8197d0'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.513856 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] The token cache is being invalidate because project 51980456e4f84b9183583477f03f3731 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81377) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:42:35.514142 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.514767 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:42:35.516223 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1470/2933] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:42:35 2022] DELETE /identity/v3/projects/51980456e4f84b9183583477f03f3731 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:42:45.181965 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:42:45.510849 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:44:10.236391 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:44:10.238028 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:44:10.299622 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:44:10.301236 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:44:10.313853 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a2361-c847-4767-adac-1f1fbaff396a admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:44:10.378678 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c06ce7-b631-4849-b968-5b38cd8197d0 admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:44:10.379147 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:44:10.379801 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] REQUEST_METHOD: `GET` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:44:10.380079 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:44:10.380382 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:44:10.382182 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:44:10.383357 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:44:10.384146 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:44:10.384484 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:44:10.384756 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] PATH_INFO: `/v3/users/7b3e6035de1e4fe3a651ece7d82842eb` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:44:10.385889 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3e6035de1e4fe3a651ece7d82842eb)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:44:10.387778 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:44:10.393817 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:44:10.395371 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:44:10.405935 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1464/2934] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 12:44:10.437057 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3e6035de1e4fe3a651ece7d82842eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.445393 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3e6035de1e4fe3a651ece7d82842eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.446095 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3e6035de1e4fe3a651ece7d82842eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.453334 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3e6035de1e4fe3a651ece7d82842eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.466287 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3e6035de1e4fe3a651ece7d82842eb'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.472605 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1471/2935] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:44:10 2022] DELETE /identity/v3/users/7b3e6035de1e4fe3a651ece7d82842eb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 12:44:10.645322 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:44:10.662246 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595be2c-ccba-4128-97a8-9cfb059da44d service placement] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:44:10.719882 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:44:10.720528 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:44:10.720808 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:44:10.721083 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] PATH_INFO: `/v3/projects/f4d530e1b20c47319f5e8949d808c1b3` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:44:10.722190 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d530e1b20c47319f5e8949d808c1b3)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:44:10.724372 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:44:10.761170 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:44:10.771900 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d530e1b20c47319f5e8949d808c1b3', 'request_id': 'req-ea9838fa-370f-4a46-a761-39bd33c123af'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.779469 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d530e1b20c47319f5e8949d808c1b3', 'request_id': 'req-ea9838fa-370f-4a46-a761-39bd33c123af'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.785845 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d530e1b20c47319f5e8949d808c1b3', 'request_id': 'req-ea9838fa-370f-4a46-a761-39bd33c123af'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.796898 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] The token cache is being invalidate because project f4d530e1b20c47319f5e8949d808c1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:44:10.797187 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.798364 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.799846 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1465/2936] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:44:10 2022] DELETE /identity/v3/projects/f4d530e1b20c47319f5e8949d808c1b3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:44:10.803818 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:44:10.805304 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:44:10.818580 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92f62-a13b-44e4-9ef3-169ae5d17dbb admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:44:10.875199 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:44:10.875862 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:44:10.876137 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:44:10.877460 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] PATH_INFO: `/v3/users/ed176c91eeee4a7aa8edd4cbb70a0514` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:44:10.878610 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed176c91eeee4a7aa8edd4cbb70a0514)` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:44:10.880545 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:44:10.902807 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:44:10.929335 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed176c91eeee4a7aa8edd4cbb70a0514'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.936634 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed176c91eeee4a7aa8edd4cbb70a0514'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.937285 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed176c91eeee4a7aa8edd4cbb70a0514'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.944233 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed176c91eeee4a7aa8edd4cbb70a0514'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.955976 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.notifications [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed176c91eeee4a7aa8edd4cbb70a0514'} {{(pid=81377) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:10.961805 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1472/2937] 10.222.0.33 () {58 vars in 1224 bytes} [Thu Sep 22 12:44:10 2022] DELETE /identity/v3/users/ed176c91eeee4a7aa8edd4cbb70a0514 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:44:11.132772 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Authenticating user token {{(pid=81376) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:44:11.134310 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.fernet_utils [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81376) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:44:11.148337 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9838fa-370f-4a46-a761-39bd33c123af admin admin] Validating token access rules against request {{(pid=81376) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:44:11.205582 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9dcd38c10c5f47bf8d47fab51d035bf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8012745f80ac4412b0055fc9da17a750', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81376) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:44:11.206209 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:44:11.206486 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:44:11.206758 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] PATH_INFO: `/v3/projects/6ac63e237a6e4a008f318c662941f173` {{(pid=81376) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:44:11.207861 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac63e237a6e4a008f318c662941f173)` {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:44:11.209890 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] RBAC: Authorization granted {{(pid=81376) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:44:11.255411 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac63e237a6e4a008f318c662941f173', 'request_id': 'req-919b5481-a114-4c1e-8c84-974a25001351'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:11.262633 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac63e237a6e4a008f318c662941f173', 'request_id': 'req-919b5481-a114-4c1e-8c84-974a25001351'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:11.269096 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac63e237a6e4a008f318c662941f173', 'request_id': 'req-919b5481-a114-4c1e-8c84-974a25001351'} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:11.280639 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] The token cache is being invalidate because project 6ac63e237a6e4a008f318c662941f173 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81376) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 12:44:11.280927 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:11.281586 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG keystone.notifications [None req-919b5481-a114-4c1e-8c84-974a25001351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81376) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 12:44:11.283657 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: [pid: 81376|app: 0|req: 1466/2938] 10.222.0.33 () {58 vars in 1233 bytes} [Thu Sep 22 12:44:11 2022] DELETE /identity/v3/projects/6ac63e237a6e4a008f318c662941f173 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 12:44:20.958362 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:44:21.277470 dv-cinder-856463-4-iscsi devstack@keystone.service[81376]: DEBUG dbcounter [-] [81376] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=81376) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 12:47:13.304888 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Authenticating user token {{(pid=81377) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 12:47:13.306300 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:47:13.441686 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4848e15-4680-4696-8cd4-6dec0c26fc8b admin admin] Validating token access rules against request {{(pid=81377) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 12:47:13.498965 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'faf0bcc4ab8649cf98889340287a227c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6084e6bd20cd4e1fa82581fec46a2f78', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81377) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 12:47:13.499560 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] REQUEST_METHOD: `GET` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 12:47:13.499836 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] SCRIPT_NAME: `/identity` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 12:47:13.500107 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81377) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 12:47:13.501853 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.fernet_utils [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81377) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 12:47:13.512668 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 12:47:13.514257 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b884d00d-abf4-4311-a9a9-dc7a53e1f0ad service placement] RBAC: Authorization granted {{(pid=81377) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 12:47:13.524132 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: [pid: 81377|app: 0|req: 1473/2939] 10.222.0.33 () {60 vars in 1365 bytes} [Thu Sep 22 12:47:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 12:47:23.519977 dv-cinder-856463-4-iscsi devstack@keystone.service[81377]: DEBUG dbcounter [-] [81377] Writing DB stats keystone:SELECT=89 {{(pid=81377) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}